diff --git a/PARITY.md b/PARITY.md index 387e2bede..2beb45b9d 100644 --- a/PARITY.md +++ b/PARITY.md @@ -257,6 +257,13 @@ the skip rules (configured `ignore` plus the built-in `node_modules/**` and by `db::query` and the `dirsql_path` module). All three SDKs and the CLI get the same answers with no per-binding surface and no SDK signature changes. +**Path-table braces follow bash brace expansion (#1262) — parity by +construction, no drift.** A comma group alternates, a `..` range expands, and +any other brace is literal text. The expansion runs in the shared Rust core +(`brace::expand`, called by `scanner::compile_glob`), so all three SDKs and the +CLI match the same files with no per-binding surface and no SDK signature +changes. Config `[[table]] glob` does not use it yet. + **Gitignore-by-default in path-tables (#742) — at parity, opt-out restored (#745/#746).** Path-table scans respect `.gitignore` files by default (hierarchical, traversal-pruned, hidden files still scanned). The *behavior* diff --git a/docs/reference/path-tables.md b/docs/reference/path-tables.md index 9c7a8da8f..52b830ef5 100644 --- a/docs/reference/path-tables.md +++ b/docs/reference/path-tables.md @@ -54,6 +54,13 @@ depth, as in the shell. A trailing `/` is `*` appended, so `'./*/'` is A path containing `*`, `?`, `[` or `{` is a glob and is used exactly as written: `*` matches within a single directory, `**` crosses directories. +Braces expand as bash expands them. A group holding a comma is an +alternation: `'./docs/{a,b}.md'` is `docs/a.md` and `docs/b.md`, and groups +nest. A group holding a `..` range counts: `'./log/{1..3}.txt'`, +`'./{a..c}.md'`, `'./{01..10}.md'` (zero-padded) and `'./{1..10..2}.md'` +(every second value). Any other brace is literal text, so `'./br/{q}.md'` +names the file `br/{q}.md`, and `\{` is always a literal brace. + The scan starts at the last directory named outright before the first glob component, and descends only as deep as the pattern can match -- `'./small/*.md'` lists `small/` and nothing else -- so a query over one diff --git a/packages/python/changelog.d/2026-10-05-brace-groups-follow-bash.md b/packages/python/changelog.d/2026-10-05-brace-groups-follow-bash.md new file mode 100644 index 000000000..27cefd4ab --- /dev/null +++ b/packages/python/changelog.d/2026-10-05-brace-groups-follow-bash.md @@ -0,0 +1,6 @@ +**Changed** Braces in a path-table follow bash brace expansion. A group with +a top-level comma (`{a,b}`, nested allowed) is an alternation, and a `..` +range (`{1..3}`, `{a..c}`, `{01..10}`, `{1..10..2}`) expands to each value. +Any other brace is literal text: `'./br/{q}.md'` names the file `br/{q}.md` +where it used to match `br/q.md`, and `{` or `}` without a partner no longer +fails to compile. Config `[[table]] glob` is unchanged. diff --git a/packages/python/e2e-attestations/claude-1262-brace-literal.json b/packages/python/e2e-attestations/claude-1262-brace-literal.json new file mode 100644 index 000000000..7e583b400 --- /dev/null +++ b/packages/python/e2e-attestations/claude-1262-brace-literal.json @@ -0,0 +1,7 @@ +{ + "command": "just test-e2e", + "ran_at": 1791240509, + "exit_code": 0, + "commit": "42886c665ab14434f259593bf7868455cf206eda", + "branch": "claude/1262-brace-literal" +} diff --git a/packages/python/migrations.d/2026-10-05-brace-groups-follow-bash.md b/packages/python/migrations.d/2026-10-05-brace-groups-follow-bash.md new file mode 100644 index 000000000..ca63b5613 --- /dev/null +++ b/packages/python/migrations.d/2026-10-05-brace-groups-follow-bash.md @@ -0,0 +1,39 @@ +### Core: path-table braces follow bash brace expansion + +#### Summary + +A path-table's braces now expand as bash expands them before globbing: a +group with a top-level comma is an alternation, a `..` range expands to each +value, and every other brace is literal text. `'./br/{q}.md'` used to match +`br/q.md` and now matches only a file named `br/{q}.md`; `'./a{1..3}.md'` +used to match nothing and now matches `a1.md`, `a2.md`, `a3.md`. Comma groups +behave as before. Every surface that resolves a path-table is affected (CLI, +REPL, HTTP server, SDKs, `--on-file`); config `[[table]] glob` is not. + +#### Required changes + +| Surface | Before | After | +| ------- | ------ | ----- | +| One-option group | `SELECT * FROM './br/{q}.md'` (matched `br/q.md`) | `SELECT * FROM './br/q.md'` | + +#### Deprecations removed + +_None._ + +#### Behavior changes without code changes + +- A brace group with neither a top-level comma nor a valid `..` range is + literal text. +- `{1..3}`, `{a..c}`, `{01..10}` and `{1..10..2}` expand to their values + instead of matching the literal text between the braces. +- An unpaired `{` or `}` is literal text instead of a glob compile error. + +#### Verification + +```python +# root holds q.md, {q}.md, a1.md and a2.md +rows = await DirSQL(root).query("SELECT path FROM './{q}.md'") +# expected: [{"path": "{q}.md"}] +rows = await DirSQL(root).query("SELECT path FROM './a{1..2}.md' ORDER BY path") +# expected: [{"path": "a1.md"}, {"path": "a2.md"}] +``` diff --git a/packages/rust/changelog.d/2026-10-05-brace-groups-follow-bash.md b/packages/rust/changelog.d/2026-10-05-brace-groups-follow-bash.md new file mode 100644 index 000000000..27cefd4ab --- /dev/null +++ b/packages/rust/changelog.d/2026-10-05-brace-groups-follow-bash.md @@ -0,0 +1,6 @@ +**Changed** Braces in a path-table follow bash brace expansion. A group with +a top-level comma (`{a,b}`, nested allowed) is an alternation, and a `..` +range (`{1..3}`, `{a..c}`, `{01..10}`, `{1..10..2}`) expands to each value. +Any other brace is literal text: `'./br/{q}.md'` names the file `br/{q}.md` +where it used to match `br/q.md`, and `{` or `}` without a partner no longer +fails to compile. Config `[[table]] glob` is unchanged. diff --git a/packages/rust/migrations.d/2026-10-05-brace-groups-follow-bash.md b/packages/rust/migrations.d/2026-10-05-brace-groups-follow-bash.md new file mode 100644 index 000000000..4fc0b91d6 --- /dev/null +++ b/packages/rust/migrations.d/2026-10-05-brace-groups-follow-bash.md @@ -0,0 +1,40 @@ +### Core: path-table braces follow bash brace expansion + +#### Summary + +A path-table's braces now expand as bash expands them before globbing: a +group with a top-level comma is an alternation, a `..` range expands to each +value, and every other brace is literal text. `'./br/{q}.md'` used to match +`br/q.md` and now matches only a file named `br/{q}.md`; `'./a{1..3}.md'` +used to match nothing and now matches `a1.md`, `a2.md`, `a3.md`. Comma groups +behave as before. Every surface that resolves a path-table is affected (CLI, +REPL, HTTP server, SDKs, `--on-file`); config `[[table]] glob` is not. + +#### Required changes + +| Surface | Before | After | +| ------- | ------ | ----- | +| One-option group | `SELECT * FROM './br/{q}.md'` (matched `br/q.md`) | `SELECT * FROM './br/q.md'` | + +#### Deprecations removed + +_None._ + +#### Behavior changes without code changes + +- A brace group with neither a top-level comma nor a valid `..` range is + literal text. +- `{1..3}`, `{a..c}`, `{01..10}` and `{1..10..2}` expand to their values + instead of matching the literal text between the braces. +- An unpaired `{` or `}` is literal text instead of a glob compile error. + +#### Verification + +```bash +cd "$(mktemp -d)" +touch q.md '{q}.md' a1.md a2.md +dirsql query "SELECT path FROM './{q}.md'" +# expected: [{"path":"{q}.md"}] +dirsql query "SELECT path FROM './a{1..2}.md' ORDER BY path" +# expected: [{"path":"a1.md"},{"path":"a2.md"}] +``` diff --git a/packages/rust/src/brace.rs b/packages/rust/src/brace.rs new file mode 100644 index 000000000..b96fe195f --- /dev/null +++ b/packages/rust/src/brace.rs @@ -0,0 +1,425 @@ +//! Bash brace expansion for path globs. +//! +//! A pattern expands the way bash expands a word before globbing it: a brace +//! group holding a top-level comma is an alternation, one holding a `..` +//! sequence is a range, and any other brace is literal text. The expansions +//! come back as globset patterns with every remaining brace escaped, so +//! globset never sees an alternation of its own. + +/// Expand `pattern`'s brace groups as bash does, one globset pattern per word. +pub(crate) fn expand(pattern: &str) -> Vec { + let chars: Vec = pattern.chars().collect(); + expand_word(&chars) + .iter() + .map(|word| escape_braces(word)) + .collect() +} + +#[derive(Clone, Copy, PartialEq)] +enum Stop { + Close, + Comma, +} + +/// The characters of `text` from `from` on that no backslash escapes, with +/// their indices; the backslashes themselves are dropped too. +fn unescaped(text: &[char], from: usize) -> impl Iterator + '_ { + let mut escaped = false; + text.iter() + .copied() + .enumerate() + .skip(from) + .filter(move |&(_, c)| { + let keep = !escaped && c != '\\'; + escaped = !escaped && c == '\\'; + keep + }) +} + +/// Index of the first unescaped `stop` at nesting level zero, scanning from +/// `from`; `text.len()` when there is none. A `}` only closes once a comma or +/// a `..` has been seen, which is why `{q}` never closes. +fn gobble(text: &[char], from: usize, stop: Stop) -> usize { + let target = match stop { + Stop::Close => '}', + Stop::Comma => ',', + }; + let mut level = 0usize; + let mut closable = stop != Stop::Close; + for (i, c) in unescaped(text, from) { + if c == target && level == 0 && closable { + return i; + } + if c == '{' { + level += 1; + } else if c == '}' && level > 0 { + level -= 1; + } else if level == 0 + && (c == ',' || (text[i..].starts_with(&['.', '.']) && text.get(i + 2) != Some(&'}'))) + { + closable = true; + } + } + text.len() +} + +/// Bash never opens a group at a `{` that starts a word-like token and is +/// followed by whitespace or `}`, so `{}` stays intact for `find -exec`. +fn is_shell_brace(text: &[char], i: usize) -> bool { + let after_space = i == 0 || text[i - 1].is_ascii_whitespace(); + let before_space_or_close = text + .get(i + 1) + .is_some_and(|c| c.is_ascii_whitespace() || *c == '}'); + after_space && before_space_or_close +} + +fn expand_word(text: &[char]) -> Vec { + let group = unescaped(text, 0) + .filter(|&(i, c)| c == '{' && !is_shell_brace(text, i)) + .find_map(|(open, _)| { + let close = gobble(text, open + 1, Stop::Close); + (close < text.len()).then_some((open, close)) + }); + let Some((open, close)) = group else { + return vec![text.iter().collect()]; + }; + let preamble: String = text[..open].iter().collect(); + let amble = &text[open + 1..close]; + let middle = if unescaped(amble, 0).any(|(_, c)| c == ',') { + expand_alternatives(amble) + } else { + let amble: String = amble.iter().collect(); + sequence(&amble).unwrap_or_else(|| vec![format!("{{{amble}}}")]) + }; + let rest = expand_word(&text[close + 1..]); + middle + .iter() + .flat_map(|m| { + rest.iter() + .map(|r| format!("{preamble}{m}{r}")) + .collect::>() + }) + .collect() +} + +fn expand_alternatives(amble: &[char]) -> Vec { + let end = gobble(amble, 0, Stop::Comma); + let mut words = expand_word(&amble[..end]); + if let Some(tail) = amble.get(end + 1..) { + words.extend(expand_alternatives(tail)); + } + words +} + +/// The `..` range bash expands `{lo..hi}` / `{lo..hi..step}` to, or `None` +/// when the text is not a range bash accepts. +fn sequence(amble: &str) -> Option> { + let (lhs, rhs) = amble.split_once("..")?; + if lhs.is_empty() || rhs.is_empty() { + return None; + } + let digits = number_prefix_len(rhs); + let (end, rhs_len) = if let Ok(n) = rhs[..digits].parse() { + (Term::Int(n), digits) + } else { + let c = rhs.as_bytes()[0]; + if !c.is_ascii_alphabetic() || !(rhs.len() == 1 || rhs.as_bytes()[1] == b'.') { + return None; + } + (Term::Char(c), 1) + }; + let step = match &rhs[rhs_len..] { + "" => 1, + tail => tail + .strip_prefix("..") + .filter(|s| !s.is_empty())? + .parse() + .ok()?, + }; + let start = if let Ok(n) = lhs.parse::() { + Term::Int(n) + } else if lhs.len() == 1 && lhs.as_bytes()[0].is_ascii_alphabetic() { + Term::Char(lhs.as_bytes()[0]) + } else { + return None; + }; + match (start, end) { + (Term::Int(lo), Term::Int(hi)) => { + let width = pad_width(lhs, &rhs[..rhs_len]); + Some( + steps(lo, hi, step)? + .map(|n| format!("{n:0width$}")) + .collect(), + ) + } + (Term::Char(lo), Term::Char(hi)) => Some( + steps(i64::from(lo), i64::from(hi), step)? + .filter_map(|n| u8::try_from(n).ok()) + .map(|b| char::from(b).to_string()) + .collect(), + ), + _ => None, + } +} + +enum Term { + Int(i64), + Char(u8), +} + +fn number_prefix_len(s: &str) -> usize { + let sign = usize::from(matches!(s.as_bytes()[0], b'+' | b'-')); + sign + s[sign..].bytes().take_while(u8::is_ascii_digit).count() +} + +/// Bash zero-pads every term to the widest end written with a leading zero. +fn pad_width(lhs: &str, rhs: &str) -> usize { + let padded = + |s: &str| (s.len() > 1 && s.starts_with('0')) || (s.len() > 2 && s.starts_with("-0")); + if padded(lhs) || padded(rhs) { + lhs.len().max(rhs.len()) + } else { + 0 + } +} + +/// `lo` to `hi` inclusive in steps of `|step|` (0 counts as 1), walking down +/// when `hi < lo`. `None` past the element count bash refuses to generate. +fn steps(lo: i64, hi: i64, step: i64) -> Option> { + let stride = i128::from(step).abs().max(1); + let distance = i128::from(hi) - i128::from(lo); + if distance.abs() / stride > i128::from(i32::MAX) - 3 { + return None; + } + let count = distance.abs() / stride + 1; + let signed = stride * distance.signum(); + Some((0..count).map(move |k| i128::from(lo) + k * signed)) +} + +/// Escape the braces bash left as literal text, so globset matches them +/// rather than reading an alternation. A complete `[...]` class is copied as +/// written; an unclosed one ends escaping so globset reports the error. +fn escape_braces(word: &str) -> String { + let chars: Vec = word.chars().collect(); + let mut out = String::with_capacity(word.len()); + let mut resume = 0; + for (i, &c) in chars.iter().enumerate() { + if i < resume { + continue; + } + match (c, chars.get(i + 1)) { + ('\\', Some(&next)) => { + match next { + '{' => out.push_str("[{]"), + '}' => out.push_str("[}]"), + ',' => out.push(','), + _ => out.extend(['\\', next]), + } + resume = i + 2; + } + ('[', _) => match class_end(&chars, i) { + Some(end) => { + out.extend(&chars[i..=end]); + resume = end + 1; + } + None => { + out.extend(&chars[i..]); + return out; + } + }, + ('{', _) => out.push_str("[{]"), + ('}', _) => out.push_str("[}]"), + _ => out.push(c), + } + } + out +} + +/// Index of the `]` closing the class globset opens at `open`, where a `]` +/// first in the class (after any `!` / `^`) is a member, not the close. +fn class_end(chars: &[char], open: usize) -> Option { + let mut i = open + 1; + if matches!(chars.get(i), Some('!' | '^')) { + i += 1; + } + if chars.get(i) == Some(&']') { + i += 1; + } + (i..chars.len()).find(|&j| chars[j] == ']') +} + +#[cfg(test)] +mod tests { + use super::*; + + fn words(pattern: &str) -> Vec { + expand_word(&pattern.chars().collect::>()) + } + + #[test] + fn a_group_without_a_comma_or_sequence_is_literal() { + for pattern in [ + "x{q}y", "x{}y", "x{a,b", "xa,b}y", "x{}}y", "x{{}}y", "{},a}", + ] { + assert_eq!(words(pattern), vec![pattern], "{pattern}"); + } + } + + #[test] + fn a_comma_group_alternates_in_order() { + assert_eq!(words("x{a,b}y"), vec!["xay", "xby"]); + assert_eq!(words("{a,b}{c,d}"), vec!["ac", "ad", "bc", "bd"]); + } + + #[test] + fn empty_alternatives_are_kept() { + assert_eq!(words("x{,a}y"), vec!["xy", "xay"]); + assert_eq!(words("x{a,}y"), vec!["xay", "xy"]); + assert_eq!(words("x{,}y"), vec!["xy", "xy"]); + } + + #[test] + fn nested_groups_expand_inside_out_as_bash_does() { + assert_eq!(words("x{a,{b,c}}y"), vec!["xay", "xby", "xcy"]); + assert_eq!(words("x{a{b,c}}y"), vec!["x{ab}y", "x{ac}y"]); + assert_eq!(words("x{{a,b}}y"), vec!["x{a}y", "x{b}y"]); + assert_eq!(words("x{a,{b}}y"), vec!["xay", "x{b}y"]); + assert_eq!(words("x{{},a}y"), vec!["x{}y", "xay"]); + assert_eq!(words("x{}a,b}y"), vec!["x}ay", "xby"]); + assert_eq!(words("x{..{a,b}}y"), vec!["x..ay", "x..by"]); + } + + #[test] + fn stray_braces_beside_a_group_are_literal() { + assert_eq!(words("x{{a,b}y"), vec!["x{ay", "x{by"]); + assert_eq!(words("x{a,b}}y"), vec!["xa}y", "xb}y"]); + assert_eq!(words("x{a,}}y"), vec!["xa}y", "x}y"]); + assert_eq!(words("xx{a..}y,b}"), vec!["xxa..}y", "xxb"]); + } + + #[test] + fn an_escaped_brace_or_comma_does_not_count() { + assert_eq!(words(r"x\{a,b}y"), vec![r"x\{a,b}y"]); + assert_eq!(words(r"x{a\,b}y"), vec![r"x{a\,b}y"]); + assert_eq!(words(r"x{a\,b,c}y"), vec![r"xa\,by", "xcy"]); + assert_eq!(words(r"x{\}a,b}y"), vec![r"x\}ay", "xby"]); + } + + #[test] + fn a_brace_before_whitespace_or_a_close_after_whitespace_never_opens() { + assert_eq!(words("x {},a}y"), vec!["x {},a}y"]); + assert_eq!(words("{ a,b}"), vec!["{ a,b}"]); + assert_eq!(words("x{ a,b}"), vec!["x a", "xb"]); + } + + #[test] + fn integer_sequences_count_either_way() { + assert_eq!(words("x{1..3}y"), vec!["x1y", "x2y", "x3y"]); + assert_eq!(words("x{3..1}"), vec!["x3", "x2", "x1"]); + assert_eq!(words("{-2..1}"), vec!["-2", "-1", "0", "1"]); + assert_eq!(words("{+1..3}"), vec!["1", "2", "3"]); + assert_eq!(words("{1..-1}"), vec!["1", "0", "-1"]); + assert_eq!(words("{1..1}"), vec!["1"]); + } + + #[test] + fn a_step_is_taken_by_magnitude() { + assert_eq!(words("{1..10..3}"), vec!["1", "4", "7", "10"]); + assert_eq!(words("{10..1..3}"), vec!["10", "7", "4", "1"]); + assert_eq!(words("{1..3..-1}"), vec!["1", "2", "3"]); + assert_eq!(words("{1..3..0}"), vec!["1", "2", "3"]); + } + + #[test] + fn a_leading_zero_pads_every_term() { + assert_eq!(words("{08..10}"), vec!["08", "09", "10"]); + assert_eq!(words("{1..03}"), vec!["01", "02", "03"]); + assert_eq!(words("{001..2}"), vec!["001", "002"]); + assert_eq!(words("{-01..1}"), vec!["-01", "000", "001"]); + assert_eq!(words("{0..10..5}"), vec!["0", "5", "10"]); + assert_eq!(words("{-0..10..5}"), vec!["0", "5", "10"]); + } + + #[test] + fn letter_sequences_walk_the_alphabet() { + assert_eq!(words("{a..c}"), vec!["a", "b", "c"]); + assert_eq!(words("{e..a..2}"), vec!["e", "c", "a"]); + assert_eq!(words("{Y..[}"), vec!["{Y..[}"]); + } + + #[test] + fn a_malformed_sequence_is_literal() { + for pattern in [ + "{1..a}", + "{ab..c}", + "{1...3}", + "{a..}", + "{..a}", + "{1..3..}", + "{ab..cd}", + "{1..3...}", + "{1..3.}", + "{1..3..x}", + "{é..f}", + "{9223372036854775807..9223372036854775808}", + ] { + assert_eq!(words(pattern), vec![pattern], "{pattern}"); + } + } + + #[test] + fn a_malformed_sequence_leaves_later_groups_expanding() { + assert_eq!(words("x{1..z}{a,b}"), vec!["x{1..z}a", "x{1..z}b"]); + assert_eq!(words("x{q}{1..2}"), vec!["x{q}1", "x{q}2"]); + } + + #[test] + fn a_comma_beside_dots_makes_an_alternation_not_a_range() { + assert_eq!(words("{a..c,d}"), vec!["a..c", "d"]); + assert_eq!(words("{1..3,}"), vec!["1..3", ""]); + } + + #[test] + fn a_range_bash_refuses_to_generate_stays_literal() { + assert!(steps(0, i64::from(i32::MAX), 1).is_none()); + assert!(steps(0, i64::from(i32::MAX) - 2, 1).is_none()); + let widest = steps(0, i64::from(i32::MAX) - 3, 1).unwrap(); + assert_eq!(widest.size_hint().0, i32::MAX as usize - 2); + assert!(steps(0, i64::from(i32::MAX), 2).is_some()); + } + + #[test] + fn a_range_missing_either_end_is_not_a_sequence() { + assert!(sequence("a..").is_none()); + assert!(sequence("..a").is_none()); + } + + #[test] + fn literal_braces_are_escaped_for_globset() { + assert_eq!(escape_braces("x{q}y"), "x[{]q[}]y"); + assert_eq!(escape_braces(r"x\{a,b\}"), "x[{]a,b[}]"); + assert_eq!(escape_braces(r"a\,b"), "a,b"); + assert_eq!(escape_braces(r"a\*b\"), r"a\*b\"); + } + + #[test] + fn a_class_is_copied_as_written() { + assert_eq!(escape_braces("[{}]x"), "[{}]x"); + assert_eq!(escape_braces("[]{]{"), "[]{][{]"); + assert_eq!(escape_braces("[!]{]{"), "[!]{][{]"); + assert_eq!(escape_braces("[^}]}"), "[^}][}]"); + } + + #[test] + fn an_unclosed_class_ends_escaping() { + assert_eq!(escape_braces("x[{q}"), "x[{q}"); + assert_eq!(escape_braces("x[]{"), "x[]{"); + } + + #[test] + fn expand_escapes_every_word() { + assert_eq!(expand("{a,{b}}.md"), vec!["a.md", "[{]b[}].md"]); + assert_eq!(expand("{q}"), vec!["[{]q[}]"]); + assert_eq!(expand("plain/*.md"), vec!["plain/*.md"]); + } +} diff --git a/packages/rust/src/lib.rs b/packages/rust/src/lib.rs index b980e4276..c787bde08 100644 --- a/packages/rust/src/lib.rs +++ b/packages/rust/src/lib.rs @@ -7,6 +7,7 @@ //! bindings in this workspace can reach them, but they are not part of the //! stable public API. +mod brace; /// Reusable command runner backing the command-backed events. pub mod command; #[doc(hidden)] diff --git a/packages/rust/src/scanner.rs b/packages/rust/src/scanner.rs index 4138a8cc2..ac8849359 100644 --- a/packages/rust/src/scanner.rs +++ b/packages/rust/src/scanner.rs @@ -157,9 +157,9 @@ fn is_wanted(glob: &PathGlob, ignore: &TableMatcher, rel_path: &Path) -> bool { } /// A path-table's glob as [`scan_glob`] reads it: the files it matches, and -/// the dot-named entries it spells out and so lets the walk into, and the -/// same pattern split into `/`-separated components, which decide where the -/// walk may follow a symlink. +/// the dot-named entries it spells out and so lets the walk into, and each +/// brace-expanded word split into `/`-separated components, which decide +/// where the walk may follow a symlink. /// /// A component beginning with `.` is spelled; `.claude` and `.env` admit /// exactly those names, `.*` any dot-named entry, as in the shell. Every @@ -173,15 +173,19 @@ fn is_wanted(glob: &PathGlob, ignore: &TableMatcher, rel_path: &Path) -> bool { /// cannot recurse without bound. #[derive(Debug)] pub struct PathGlob { - files: Pattern, + files: Vec, spelled_dot_names: Vec, + /// Every word's components back to back, each word closed by an `End`. components: Vec, + /// Where each word's components begin. + starts: Vec, } #[derive(Debug)] enum Component { AnyDepth, Name(Pattern), + End, } #[derive(Debug, Clone, Copy, PartialEq)] @@ -199,7 +203,7 @@ impl PathGlob { } pub fn is_match(&self, rel_path: &Path) -> bool { - self.files.is_match(rel_path) + self.files.iter().any(|p| p.is_match(rel_path)) } fn has_components(&self) -> bool { @@ -207,11 +211,11 @@ impl PathGlob { } fn start(&self) -> Vec { - self.closure(vec![0]) + self.closure(self.starts.clone()) } /// The components that may come next once `name` has been consumed from - /// `states`; `components.len()` means the whole pattern has matched. + /// `states`; an `End` means a whole word has matched. fn step(&self, states: &[usize], name: &OsStr, kind: Kind) -> Vec { let mut next = Vec::new(); for &i in states { @@ -243,11 +247,11 @@ impl PathGlob { /// Whether an entry below a followed symlink is worth taking: a file the /// whole pattern matches, or a directory some component can still enter. fn reaches(&self, states: &[usize], kind: Kind) -> bool { - let end = self.components.len(); + let at_end = |&i: &usize| matches!(self.components[i], Component::End); if kind == Kind::File { - states.contains(&end) + states.iter().any(at_end) } else { - states.iter().any(|&i| i < end) + !states.iter().all(at_end) } } } @@ -258,15 +262,30 @@ impl PathGlob { /// it a lone `*` would cross `/` and the explicit non-recursive spelling would /// silently recurse. `**` still crosses separators. pub fn compile_glob(pattern: &str) -> Result { - let spelled_dot_names = pattern - .split('/') - .filter(|c| is_dot_named(OsStr::new(c))) - .map(Pattern::new) - .collect::>()?; + let words = crate::brace::expand(pattern); + let mut files = Vec::new(); + let mut spelled_dot_names = Vec::new(); + for word in &words { + files.push(Pattern::new(word)?); + for component in word.split('/').filter(|c| is_dot_named(OsStr::new(c))) { + spelled_dot_names.push(Pattern::new(component)?); + } + } + let mut components = Vec::new(); + let mut starts = Vec::new(); + // All or nothing: the walk prunes a directory no component can enter, so + // a word left without components would lose its matches. + let compiled: Option> = words.iter().map(|w| compile_components(w)).collect(); + for word_components in compiled.unwrap_or_default() { + starts.push(components.len()); + components.extend(word_components); + components.push(Component::End); + } Ok(PathGlob { - files: Pattern::new(pattern)?, + files, spelled_dot_names, - components: compile_components(pattern), + components, + starts, }) } @@ -274,19 +293,19 @@ fn is_dot_named(name: &OsStr) -> bool { name.as_encoded_bytes().first() == Some(&b'.') } -/// A pattern whose components do not compile on their own (an alternation -/// spanning a `/`) gets none, so the walk follows no symlinked directory. -fn compile_components(pattern: &str) -> Vec { - let mut components = pattern +/// A word whose components do not compile on their own (a class spanning a +/// `/`) gets none. +fn compile_components(word: &str) -> Option> { + let mut components = word .split('/') .map(|c| match c { "**" => Ok(Component::AnyDepth), _ => Pattern::new(c).map(Component::Name), }) .collect::, _>>() - .unwrap_or_default(); + .ok()?; components.dedup_by(|a, b| matches!((a, b), (Component::AnyDepth, Component::AnyDepth))); - components + Some(components) } /// Module-argument spelling for a scan that respects `.gitignore`. @@ -963,18 +982,51 @@ mod tests { #[test] fn repeated_double_stars_are_one() { let glob = compile_glob("**/**/x").unwrap(); - assert_eq!(glob.components.len(), 2); + assert_eq!(glob.components.len(), 3); assert_eq!(states_after(&glob, &[("a", Kind::LinkedDir)]), vec![1]); } #[test] fn a_pattern_whose_components_do_not_compile_alone_follows_no_link() { - let glob = compile_glob("{a,b/c}").unwrap(); - assert!(glob.is_match(Path::new("b/c"))); + let glob = compile_glob("[a/b]").unwrap(); assert!(glob.components.is_empty()); assert!(!glob.reaches(&glob.start(), Kind::LinkedDir)); } + #[test] + fn one_brace_word_without_components_leaves_the_glob_without_any() { + let glob = compile_glob("{x,[a/b]}").unwrap(); + assert!(glob.components.is_empty()); + assert!(!glob.has_components()); + } + + #[test] + fn each_brace_word_is_walked_from_its_own_start() { + let glob = compile_glob("{a,b/c}").unwrap(); + assert!(glob.is_match(Path::new("b/c"))); + assert_eq!(glob.start(), vec![0, 2]); + assert_eq!(states_after(&glob, &[("b", Kind::LinkedDir)]), vec![3]); + let states = states_after(&glob, &[("b", Kind::LinkedDir), ("c", Kind::File)]); + assert!(glob.reaches(&states, Kind::File)); + let states = states_after(&glob, &[("a", Kind::File)]); + assert!(glob.reaches(&states, Kind::File)); + assert!(!glob.reaches(&states, Kind::LinkedDir)); + } + + #[test] + fn a_literal_brace_component_matches_only_its_own_text() { + let glob = compile_glob("{q}/x").unwrap(); + assert_eq!(states_after(&glob, &[("{q}", Kind::LinkedDir)]), vec![1]); + assert!(states_after(&glob, &[("q", Kind::LinkedDir)]).is_empty()); + } + + #[test] + fn a_dot_name_inside_a_brace_group_is_spelled() { + let glob = compile_glob("{.env,x}").unwrap(); + assert!(glob.spells_dot_name(OsStr::new(".env"))); + assert!(!glob.spells_dot_name(OsStr::new(".git"))); + } + #[test] fn reaches_a_file_only_once_the_whole_pattern_has_matched() { let glob = compile_glob("*/x").unwrap(); @@ -1043,7 +1095,7 @@ mod tests { #[test] fn enters_every_real_directory_when_the_glob_has_no_components() { let ignore = TableMatcher::new(&[], &[]).unwrap(); - let glob = compile_glob("{a/b,c}").unwrap(); + let glob = compile_glob("[a/b]").unwrap(); let walk = Walk { ignore: &ignore, glob: Some(&glob), diff --git a/packages/rust/tests/path_table_cli_e2e.rs b/packages/rust/tests/path_table_cli_e2e.rs index c2d081a40..e99c579a3 100644 --- a/packages/rust/tests/path_table_cli_e2e.rs +++ b/packages/rust/tests/path_table_cli_e2e.rs @@ -399,3 +399,28 @@ fn a_posix_character_class_matches_like_bash() { assert_eq!(paths(&out), vec!["1.md", "5.md"]); } + +fn braces() -> TempDir { + let root = TempDir::new().unwrap(); + fs::create_dir_all(root.path().join("br")).unwrap(); + for name in ["q.md", "{q}.md", "a1.md", "a2.md", "a3.md"] { + fs::write(root.path().join("br").join(name), name).unwrap(); + } + root +} + +#[test] +fn a_brace_group_without_a_comma_matches_literally() { + let dir = braces(); + let out = run(&dir, "SELECT path FROM './br/{q}.md'"); + + assert_eq!(paths(&out), vec!["br/{q}.md"]); +} + +#[test] +fn a_brace_sequence_expands_like_bash() { + let dir = braces(); + let out = run(&dir, "SELECT path FROM './br/a{1..3..2}.md'"); + + assert_eq!(paths(&out), vec!["br/a1.md", "br/a3.md"]); +} diff --git a/packages/rust/tests/path_table_globs.rs b/packages/rust/tests/path_table_globs.rs index 738d4c0ac..c391920d8 100644 --- a/packages/rust/tests/path_table_globs.rs +++ b/packages/rust/tests/path_table_globs.rs @@ -800,3 +800,58 @@ fn a_negated_class_bracket_matches_one_non_ascii_character() { .is_empty() ); } + +/// Files whose names a brace group could plausibly expand to, including one +/// whose name holds the braces themselves. +fn brace_fixture() -> TempDir { + let root = TempDir::new().unwrap(); + fs::create_dir_all(root.path().join("br")).unwrap(); + for name in ["q.md", "{q}.md", "a1.md", "a2.md", "a3.md", "a{1..2}.md"] { + fs::write(root.path().join("br").join(name), name).unwrap(); + } + root +} + +#[test] +fn a_brace_group_without_a_comma_is_literal_text() { + let root = brace_fixture(); + let db = open(&root); + + assert_eq!( + paths(&db, "SELECT path FROM './br/{q}.md'"), + vec!["br/{q}.md"] + ); +} + +#[test] +fn a_brace_group_with_a_comma_is_an_alternation() { + let root = brace_fixture(); + let db = open(&root); + + assert_eq!( + paths(&db, "SELECT path FROM './br/{q,a1}.md'"), + vec!["br/a1.md", "br/q.md"] + ); +} + +#[test] +fn a_brace_sequence_expands_to_each_value() { + let root = brace_fixture(); + let db = open(&root); + + assert_eq!( + paths(&db, "SELECT path FROM './br/a{1..2}.md'"), + vec!["br/a1.md", "br/a2.md"] + ); +} + +#[test] +fn a_literal_brace_group_nested_in_an_alternation_stays_literal() { + let root = brace_fixture(); + let db = open(&root); + + assert_eq!( + paths(&db, "SELECT path FROM './br/{a3,{q}}.md'"), + vec!["br/a3.md", "br/{q}.md"] + ); +} diff --git a/packages/ts/changelog.d/2026-10-05-brace-groups-follow-bash.md b/packages/ts/changelog.d/2026-10-05-brace-groups-follow-bash.md new file mode 100644 index 000000000..27cefd4ab --- /dev/null +++ b/packages/ts/changelog.d/2026-10-05-brace-groups-follow-bash.md @@ -0,0 +1,6 @@ +**Changed** Braces in a path-table follow bash brace expansion. A group with +a top-level comma (`{a,b}`, nested allowed) is an alternation, and a `..` +range (`{1..3}`, `{a..c}`, `{01..10}`, `{1..10..2}`) expands to each value. +Any other brace is literal text: `'./br/{q}.md'` names the file `br/{q}.md` +where it used to match `br/q.md`, and `{` or `}` without a partner no longer +fails to compile. Config `[[table]] glob` is unchanged. diff --git a/packages/ts/e2e-attestations/claude-1262-brace-literal.json b/packages/ts/e2e-attestations/claude-1262-brace-literal.json new file mode 100644 index 000000000..1d04e2332 --- /dev/null +++ b/packages/ts/e2e-attestations/claude-1262-brace-literal.json @@ -0,0 +1,7 @@ +{ + "command": "pnpm test:e2e", + "ran_at": 1791240545, + "exit_code": 0, + "commit": "f6ce938426f1c162ad69d5c52254660d2f0f416d", + "branch": "claude/1262-brace-literal" +} diff --git a/packages/ts/migrations.d/2026-10-05-brace-groups-follow-bash.md b/packages/ts/migrations.d/2026-10-05-brace-groups-follow-bash.md new file mode 100644 index 000000000..aa4fd8b87 --- /dev/null +++ b/packages/ts/migrations.d/2026-10-05-brace-groups-follow-bash.md @@ -0,0 +1,39 @@ +### Core: path-table braces follow bash brace expansion + +#### Summary + +A path-table's braces now expand as bash expands them before globbing: a +group with a top-level comma is an alternation, a `..` range expands to each +value, and every other brace is literal text. `'./br/{q}.md'` used to match +`br/q.md` and now matches only a file named `br/{q}.md`; `'./a{1..3}.md'` +used to match nothing and now matches `a1.md`, `a2.md`, `a3.md`. Comma groups +behave as before. Every surface that resolves a path-table is affected (CLI, +REPL, HTTP server, SDKs, `--on-file`); config `[[table]] glob` is not. + +#### Required changes + +| Surface | Before | After | +| ------- | ------ | ----- | +| One-option group | `SELECT * FROM './br/{q}.md'` (matched `br/q.md`) | `SELECT * FROM './br/q.md'` | + +#### Deprecations removed + +_None._ + +#### Behavior changes without code changes + +- A brace group with neither a top-level comma nor a valid `..` range is + literal text. +- `{1..3}`, `{a..c}`, `{01..10}` and `{1..10..2}` expand to their values + instead of matching the literal text between the braces. +- An unpaired `{` or `}` is literal text instead of a glob compile error. + +#### Verification + +```ts +// root holds q.md, {q}.md, a1.md and a2.md +const literal = await db.query("SELECT path FROM './{q}.md'"); +// expected: [{ path: "{q}.md" }] +const range = await db.query("SELECT path FROM './a{1..2}.md' ORDER BY path"); +// expected: [{ path: "a1.md" }, { path: "a2.md" }] +``` diff --git a/testing-conventions.toml b/testing-conventions.toml index 9fadcb54b..7b3f00b89 100644 --- a/testing-conventions.toml +++ b/testing-conventions.toml @@ -108,19 +108,19 @@ features = ["cli"] [[rust.exempt]] path = "src/scanner.rs" rules = ["mutation"] -lines = [72, 131, 330, 367, 446, 467] +lines = [72, 131, 349, 386, 465, 486] reason = "scan_below, scan_glob, walk, descend, kind_of and sorted_entries read the filesystem; tests/scanner.rs (scan_finds_matching_files, scan_recurses_into_subdirectories), tests/vtab.rs (glob_scopes_the_walk, rowids_count_up_from_zero_in_scan_order) and tests/path_table_symlinks.rs assert what they return" [[rust.exempt]] path = "src/scanner.rs" rules = ["mutation"] -lines = [384, 385, 388, 447, 450] +lines = [403, 404, 407, 466, 469] reason = "whether a walked entry is a symlink, and whether the walk is below one, is known only from a real tree; tests/path_table_symlinks.rs asserts every outcome against bash, and the decisions themselves are unit-tested in classify, Walk::follows and PathGlob::step" [[rust.exempt]] path = "src/scanner.rs" rules = ["mutation"] -lines = [370, 372, 404, 481, 494] +lines = [389, 391, 423, 500, 513] reason = "whether a repo encloses the walk, and which .gitignore files are in force, is known only from a real tree holding .git; the choice of directories is unit-tested in enclosing_repo and dirs_above, and tests/path_table_gitignore.rs (with_no_repo_a_gitignore_does_not_apply_at_the_index_root, inside_a_repo_the_same_gitignore_hides_everything, a_repo_root_gitignore_above_the_index_root_applies) asserts every outcome" [[rust.exempt]] @@ -132,7 +132,7 @@ reason = "the walk's running file count is observable only through the CLI's pro [[rust.exempt]] path = "src/scanner.rs" rules = ["mutation"] -lines = [378] +lines = [397] reason = "a child's depth decides the reserved top-level .dirsql rule for a real tree; tests/scanner.rs scan_excludes_top_level_dirsql_directory and tests/vtab.rs reserved_dirsql_directory_is_skipped assert it" [[rust.exempt]]