From bc34ff3a7fb8e80ed7821b6de157abf9a8b0d185 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 09:47:50 +0000 Subject: [PATCH 01/46] feat(hooks): deny git checkout/restore that would discard uncommitted work hooks/gitapex_check_bash_safety.py's classify() extracts every path a git checkout/git restore invocation could discard, purely from token shape (no live I/O): the -- anchored form, git checkout . or a multi-positional checkout with no --, and a case-sensitive git restore flag walk (--staged/--worktree tracked separately so -S/-s are never conflated). A dynamic path token that cannot be soundly resolved to a literal denies outright, as does a segment where the classifier cannot tell which working tree is at risk (-C/--git-dir/--work-tree, a GIT_DIR=/GIT_WORK_TREE=/GIT_INDEX_FILE= assignment, or an earlier cd in the same command). hooks/check-bash-safety.sh adds a new wrapper step, structured like the existing git-push/provenance-scan step: when checkout_restore_paths is non-empty, it reads .cwd from the PreToolUse payload itself (not CLAUDE_PROJECT_DIR, which does not track the session's own cd calls) and denies if `git diff --quiet` reports any of the paths dirty against HEAD (or the empty-tree hash on an unborn HEAD). Closes the near-miss in #1128 repair 4, where `git checkout -- PATH` silently discarded an uncommitted refactor with no warning. Refs #1128. Fixes #1375. --- .gitapex/ssot.json | 7 +- hooks/check-bash-safety.sh | 61 ++ hooks/gitapex_check_bash_safety.py | 409 +++++++++++++- ...st_gitapex_check_bash_safety_properties.py | 521 +++++++++++++++++- 4 files changed, 951 insertions(+), 47 deletions(-) diff --git a/.gitapex/ssot.json b/.gitapex/ssot.json index 83371e05..e33b8550 100644 --- a/.gitapex/ssot.json +++ b/.gitapex/ssot.json @@ -84,7 +84,7 @@ "id": "bash-cli-write-and-install-guard", "kind": "script", "script": "hooks/check-bash-safety.sh", - "rule": "Delegates classification to hooks/gitapex_check_bash_safety.py, a token-based classifier (Python stdlib shlex, POSIX mode) that matches against bash's own dequoted, operator-segmented token stream -- closing the quote-splitting, ${IFS}/$IFS substitution, and variable/array/positional-parameter indirection bypass classes a prior raw-text regex substring scan was live-confirmed vulnerable to (issue #1326). Denies a Bash call matching a package/plugin-install verb (uv add/uv remove and apm install/apm uninstall stay allowed as declarative, visibly-mutating commands); denies gh issue/gh pr write subcommands and gh api writes (POST/PUT/PATCH/DELETE, a field flag, or 'mutation' in gh api graphql); on git push (including an obfuscated/indirected one that still resolves to git push), runs skills/outward-artifact-preflight/scripts/gitapex_scan_provenance.py against the outgoing commit range and warns (never blocks) if it flags something. Fails closed (denies) rather than allowing the call through when jq or python3 is missing from PATH, the payload or tool_input is not a JSON object, tool_name is present but not a string, tool_input.command is present but not a string, the classifier exits non-zero, or the classifier's own output is not a JSON object with a recognized decision. Disclosed residual (not closed by this classifier): verb-token-splitting that never places the tool/verb name as its own literal token anywhere, e.g. string-slice reconstruction (cmd=uvinstall; eval \"${cmd:0:2} ${cmd:2}\") or array-literal-assignment indirection (A=(uv); V=(install); \"${A[@]}\" \"${V[@]}\") -- see hooks/gitapex_check_bash_safety.py's own module docstring.", + "rule": "Delegates classification to hooks/gitapex_check_bash_safety.py, a token-based classifier (Python stdlib shlex, POSIX mode) that matches against bash's own dequoted, operator-segmented token stream -- closing the quote-splitting, ${IFS}/$IFS substitution, and variable/array/positional-parameter indirection bypass classes a prior raw-text regex substring scan was live-confirmed vulnerable to (issue #1326). Denies a Bash call matching a package/plugin-install verb (uv add/uv remove and apm install/apm uninstall stay allowed as declarative, visibly-mutating commands); denies gh issue/gh pr write subcommands and gh api writes (POST/PUT/PATCH/DELETE, a field flag, or 'mutation' in gh api graphql); on git push (including an obfuscated/indirected one that still resolves to git push), runs skills/outward-artifact-preflight/scripts/gitapex_scan_provenance.py against the outgoing commit range and warns (never blocks) if it flags something. Since issue #1375: the classifier also extracts every path a git checkout/git restore invocation could discard (git checkout -- PATH, git checkout ./.., git checkout with 2+ positionals and no --, or git restore PATH/--staged --worktree PATH, a case-sensitive enumerated restore flag vocabulary), denying outright (no live I/O) when a path token is dynamic and cannot be soundly resolved to a literal, when an otherwise-risky shape yields zero paths (e.g. bare 'git checkout --'), when a restore segment carries --pathspec-from-file/--pathspec-file-nul or an unrecognized flag, or when the classifier cannot soundly determine which working tree is at risk (a -C/--git-dir/--work-tree flag, a GIT_DIR=/GIT_WORK_TREE=/GIT_INDEX_FILE= assignment, or an earlier literal cd in the same command); when paths are extracted, hooks/check-bash-safety.sh reads .cwd from the PreToolUse payload itself (not $CLAUDE_PROJECT_DIR) and denies if `git -C \"$cwd\" diff --quiet -- \"$path\"` reports any of them dirty. Fails closed (denies) rather than allowing the call through when jq or python3 is missing from PATH, the payload or tool_input is not a JSON object, tool_name is present but not a string, tool_input.command is present but not a string, the classifier exits non-zero, the classifier's own output is not a JSON object with a recognized decision, .cwd is missing/not a git working tree, or a checkout/restore path's own live git-diff check cannot be verified. Disclosed residual (not closed by this classifier): verb-token-splitting that never places the tool/verb name as its own literal token anywhere, e.g. string-slice reconstruction (cmd=uvinstall; eval \"${cmd:0:2} ${cmd:2}\") or array-literal-assignment indirection (A=(uv); V=(install); \"${A[@]}\" \"${V[@]}\") -- see hooks/gitapex_check_bash_safety.py's own module docstring. A bare 'git checkout SOMENAME' (single positional, not '.'/'..', no --) is a deliberate Non-goal: disambiguating a branch/ref name from a path needs a live ref-existence lookup this pure classifier does not perform.", "planes": ["pretooluse"], "local_exclusion": "PreToolUse-only: grades a Claude Code tool-call JSON payload arriving on stdin, which has no working-tree equivalent to reconstruct ahead of a push.", "trigger": "PreToolUse matcher Bash (hooks/hooks.json)", @@ -101,7 +101,10 @@ {"kind": "bash-pattern", "ref": "gh api -X|--method POST|PUT|PATCH|DELETE"}, {"kind": "bash-pattern", "ref": "gh api -f|-F|--field|--raw-field"}, {"kind": "bash-pattern", "ref": "gh api graphql ... mutation"}, - {"kind": "bash-pattern", "ref": "git push"} + {"kind": "bash-pattern", "ref": "git push"}, + {"kind": "bash-pattern", "ref": "git checkout -- PATH..."}, + {"kind": "bash-pattern", "ref": "git checkout ."}, + {"kind": "bash-pattern", "ref": "git restore [FLAGS] PATH..."} ] }, { diff --git a/hooks/check-bash-safety.sh b/hooks/check-bash-safety.sh index 1488af5c..ff4b024c 100755 --- a/hooks/check-bash-safety.sh +++ b/hooks/check-bash-safety.sh @@ -200,4 +200,65 @@ if [ "$is_git_push" = "true" ]; then fi fi +# --- Finding 5: git checkout/restore gated on a live git-diff check (issue #1375) --- +# `git checkout -- PATH` / `git restore PATH` / `git checkout .` can discard +# uncommitted work on a tracked path with no warning at all. gitapex_check_bash_safety.py's +# own classifier already extracted every candidate path such an invocation +# could discard (its own "git checkout/restore path extraction" section), +# soundly and with no live git call of its own. Unlike Finding 4's own +# advisory provenance scan (surfaces candidates, does not decide -- warn, +# not deny), "does git diff report a difference at this path" is a binary, +# deterministic fact about repo state with no judgment-call axis, so a hit +# here denies. +checkout_restore_paths_count=$(printf '%s' "$classifier_output" | jq -r '.checkout_restore_paths | length // 0') +if [ "$checkout_restore_paths_count" -gt 0 ]; then + # Read `.cwd` from the ORIGINAL tool-call payload, not + # `${CLAUDE_PROJECT_DIR:-$(pwd)}` the way Finding 4 above does -- a push + # is not cwd-relative, but a `git diff -- PATH` pathspec check is, and + # `.cwd` is Claude Code's own record of the Bash tool call's actual + # working directory (updated on every `cd` the session runs), not this + # hook runner's own. Replaying the near-miss's own exact command from a + # subdirectory must resolve the pathspec against the SAME tree bash + # itself would use. + cwd=$(printf '%s' "$input" | jq -r '.cwd // empty') + if [ -z "$cwd" ]; then + deny "Blocked by hooks/check-bash-safety.sh: this git checkout/restore command needs the PreToolUse payload's own .cwd field to check the right working tree against, but it was missing or empty. Failing closed." + fi + if ! git -C "$cwd" rev-parse --show-toplevel >/dev/null 2>&1; then + deny "Blocked by hooks/check-bash-safety.sh: '$cwd' is not inside a git working tree -- cannot verify this git checkout/restore command is safe. Failing closed." + fi + # A fresh repo with no commits yet has no HEAD to diff against + # (`git diff --quiet HEAD -- PATH` fails with "fatal: bad revision + # 'HEAD'", confirmed live, exit 128, not the "differs" exit 1) -- + # compare against git's own well-known empty-tree object instead, so a + # genuinely clean fresh repo is not denied outright. + if git -C "$cwd" rev-parse --verify -q HEAD >/dev/null 2>&1; then + diff_base="HEAD" + else + diff_base="4b825dc642cb6eb9a060e54bf8d69288fbee4904" + fi + # Fed via process substitution (`< <(...)`), not a pipe + # (`... | while read`) -- bash runs a pipe's right-hand side in a + # subshell, where `deny`'s own `exit 2` would only exit that subshell, + # letting this script fall through to its own `exit 0` past the loop + # instead of actually denying. Process substitution keeps the loop in + # THIS shell, so `exit 2` inside it really does exit the whole script. + while IFS= read -r encoded_path; do + [ -z "$encoded_path" ] && continue + # Each line is one base64-encoded path (issue #1375: a genuine JSON + # array from the classifier, base64-encoded here too) -- a path + # containing a newline or other shell-hazardous byte would otherwise + # split across `read` calls or corrupt this loop's own field + # splitting; base64 has neither. + path=$(printf '%s' "$encoded_path" | base64 -d) + diff_exit=0 + git -C "$cwd" diff --quiet "$diff_base" -- "$path" || diff_exit=$? + if [ "$diff_exit" -eq 1 ]; then + deny "Blocked by hooks/check-bash-safety.sh: this git checkout/restore command would discard uncommitted changes at '$path'. Stash first (git stash push -- '$path') if this is not resolving a merge conflict; if it is, resolve and git add the path, or use git checkout -m -- '$path' to regenerate conflict markers instead of discarding them." + elif [ "$diff_exit" -ne 0 ]; then + deny "Blocked by hooks/check-bash-safety.sh: could not verify whether '$path' has uncommitted changes (git diff exited $diff_exit). Failing closed." + fi + done < <(printf '%s' "$classifier_output" | jq -r '.checkout_restore_paths[] | @base64') +fi + exit 0 diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 5cc17719..a9b72cfc 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -743,7 +743,7 @@ def _is_unresolvable_substitution(token: str) -> bool: return "$(" in token or "`" in token -def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, bool]: +def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `$(...)` command-substitution span's OWN inner content through this module's full rule set -- bash genuinely RUNS that inner text as a complete command the instant the @@ -778,18 +778,18 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b classifying the span's own inner content directly, instead of requiring the outer, now-opaque token to itself carry the phrase. - Returns `(reason_or_None, is_git_push)` -- ALWAYS a tuple, never a - bare `None` -- this module's own `Verdict` carries a THIRD, warn-only - `is_git_push` field the task-scoped sibling module's `Verdict` does - not, and `git push` alone is WARN-only here (`deny=False, - is_git_push=True`), not a hard deny. An earlier version of this - function only returned `is_git_push` alongside a DENY, discarding it - whenever the inner substitution's own verdict was `deny=False` -- - found live by Step 8 independent review, fifteenth round (issue - #1326): `x=$(git push origin main)` (confirmed live end-to-end - through the real hook entrypoint) silently dropped the warn signal - entirely, since folding made `git`/`push`/`origin`/`main` one opaque - token invisible to `_is_git_push_segment`'s own scan, and the + Returns `(reason_or_None, is_git_push, checkout_restore_paths)` -- + ALWAYS a 3-tuple, never a bare `None` -- this module's own `Verdict` + carries a THIRD, warn-only `is_git_push` field the task-scoped sibling + module's `Verdict` does not, and `git push` alone is WARN-only here + (`deny=False, is_git_push=True`), not a hard deny. An earlier version + of this function only returned `is_git_push` alongside a DENY, + discarding it whenever the inner substitution's own verdict was + `deny=False` -- found live by Step 8 independent review, fifteenth + round (issue #1326): `x=$(git push origin main)` (confirmed live + end-to-end through the real hook entrypoint) silently dropped the warn + signal entirely, since folding made `git`/`push`/`origin`/`main` one + opaque token invisible to `_is_git_push_segment`'s own scan, and the recursive check's own early-return-only-on-deny never propagated the inner `classify()`/`_classify_tokens()` call's OWN `is_git_push=True` result outward. This gated the outward-artifact-preflight provenance @@ -800,6 +800,13 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b each inner verdict's own `is_git_push` into a running total, returned regardless of whether any span was itself denied. + Issue #1375 threads `checkout_restore_paths` through this exact same + shape: every span's own inner `checkout_restore_paths` is concatenated + into a running tuple, unconditionally, so `x=$(git checkout -- f.py)` + is not silently dropped the same way the fifteenth round's own + `is_git_push` bug would have dropped it -- the identical bug class, + for a tuple instead of a bool. + Disclosed residual (found live by Step 8 independent review, nineteenth round, issue #1326): unlike `_rule_array_literal_content`'s own nineteenth-round fix, the recursive `_classify_tokens(inner_ @@ -813,6 +820,7 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b the finding that prompted `_rule_array_literal_content`'s own fix warranted.""" is_git_push = False + checkout_restore_paths: list[str] = [] i = 0 n = len(tokens) while i < n: @@ -840,9 +848,10 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b if inner_text.strip(): inner_verdict = classify(inner_text) is_git_push = is_git_push or inner_verdict.is_git_push + checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) if inner_verdict.deny: reason = f"a command substitution $(...) embeds a denied command -- {inner_verdict.reason}" - return reason, is_git_push + return reason, is_git_push, tuple(checkout_restore_paths) search_from = end if found_fused: i += 1 @@ -853,13 +862,14 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b if inner_tokens: inner_verdict = _classify_tokens(inner_tokens) is_git_push = is_git_push or inner_verdict.is_git_push + checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) if inner_verdict.deny: reason = f"a command substitution $(...) embeds a denied command -- {inner_verdict.reason}" - return reason, is_git_push + return reason, is_git_push, tuple(checkout_restore_paths) i = span_end continue i += 1 - return None, is_git_push + return None, is_git_push, tuple(checkout_restore_paths) def tokenize(command: str) -> list[str]: @@ -1565,7 +1575,7 @@ def _strip_leading_unassigned_bare_refs(tokens: list[str], name_to_raw_value: di def _rule_array_literal_content( tokens: list[str], name_to_value: dict[str, str], name_to_raw_value: dict[str, str] -) -> tuple[str | None, bool]: +) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `NAME=(...)` array-literal span's OWN inner content through this module's full rule set -- bash genuinely expands `"${NAME[@]}"` into that content as real argv the instant the @@ -1639,6 +1649,7 @@ def _rule_array_literal_content( for the quoted/fused `$(...)` shape) to also accept an outer scope, a larger change than this round's own confirmed finding warranted.""" is_git_push = False + checkout_restore_paths: list[str] = [] i = 0 n = len(tokens) while i < n: @@ -1655,11 +1666,12 @@ def _rule_array_literal_content( for reading, suffix in readings: reading_verdict = _classify_tokens(reading, name_to_value, name_to_raw_value) is_git_push = is_git_push or reading_verdict.is_git_push + checkout_restore_paths.extend(reading_verdict.checkout_restore_paths) if reading_verdict.deny: reason = f"an array literal NAME=(...) embeds a denied command{suffix} -- {reading_verdict.reason}" - return reason, is_git_push + return reason, is_git_push, tuple(checkout_restore_paths) i = end - return None, is_git_push + return None, is_git_push, tuple(checkout_restore_paths) # --- Denylists ----------------------------------------------------------- @@ -1777,6 +1789,15 @@ class Verdict(NamedTuple): deny: bool reason: str is_git_push: bool + # Issue #1375: every path a `git checkout`/`git restore` invocation in + # this command could discard, extracted soundly with no live I/O (see + # the "git checkout/restore path extraction" section below). Defaults + # to `()` -- every pre-existing `Verdict(...)` call site in this module + # denies for a reason unrelated to checkout/restore, where this field + # is never read (hooks/check-bash-safety.sh's own new wrapper step, + # like its existing `is_git_push` step, only ever runs on the "allow" + # decision), so none of them need updating for this new field. + checkout_restore_paths: tuple[str, ...] = () def _rule_a_literal(segments: list[list[str]]) -> str | None: @@ -2460,6 +2481,321 @@ def _is_git_push_segment(seg: list[str], name_to_raw_value: dict[str, str]) -> b return any("git push" in lit for lit in (t.lower() for t in seg if not _is_dynamic(t))) +# --- git checkout/restore path extraction (issue #1375) -------------------- +# `git checkout -- PATH` / `git restore PATH` / `git checkout .` can discard +# uncommitted work on a tracked path with no warning (the near-miss issue +# #1375 documents, issue #1128 repair 4). `classify()` stays I/O-free (this +# module's own established architecture, see `_is_git_push_segment`'s own +# `is_git_push`-then-live-wrapper-check split): this section only extracts +# every candidate path a checkout/restore invocation *could* discard, +# soundly and with no live git call, for hooks/check-bash-safety.sh's own +# new wrapper step to check against the real working tree via `git diff +# --quiet HEAD -- PATH`. An unresolved or unresolvable dynamic path token +# denies outright HERE rather than being passed through empty-handed -- +# `git diff --quiet HEAD -- PATH` exits 0 (clean) for a path that does not +# exist, so treating an unresolved token as "nothing to check" would be +# fail-OPEN, not fail-closed (confirmed live, git 2.43.0). + +_GIT_TREE_RELOCATION_LONG_FLAGS = {"--git-dir", "--work-tree"} +_GIT_GLOBAL_SHORT_VALUE_FLAGS = {"-c", "-C"} +_GIT_TREE_ENV_VARS = ("GIT_DIR", "GIT_WORK_TREE", "GIT_INDEX_FILE") + +_RESTORE_BOOLEAN_FLAGS = { + "--quiet", + "-q", + "--progress", + "--no-progress", + "--overlay", + "--no-overlay", + "--ours", + "--theirs", + "--merge", + "-m", + "--ignore-unmerged", + "--ignore-skip-worktree-bits", +} +_RESTORE_VALUE_FLAGS = {"--source", "-s", "--conflict"} + + +def _resolve_path_tokens(tokens: list[str], name_to_raw_value: dict[str, str]) -> tuple[str | None, tuple[str, ...]]: + """Resolve every token in TOKENS to one or more literal path + candidates for a `git checkout`/`git restore` invocation. A literal + token is used as-is. A dynamic token is resolved via this module's + existing `_substitute_var_refs_candidates`, called with the + CASE-PRESERVING NAME_TO_RAW_VALUE as both its value map and its + raw-value map -- unlike every other caller of that function in this + module (which resolves against the lowercased `name_to_value`, since + they only ever compare a resolved value case-insensitively against a + known tool/verb/flag literal), a filesystem path is case-sensitive, so + lowercasing it here would resolve to the wrong path. An unresolvable + token (empty candidate list), a too-large candidate set (`None`, the + same fail-closed convention `_substitute_var_refs_candidates`'s own + callers already use), or a candidate that ITSELF still contains `$`/ + backtick after substitution all deny outright here rather than being + passed to the live wrapper check empty-handed -- see this section's + own module-level comment for why that would be fail-open. + + The still-dynamic-candidate check closes a real gap found while + building this function: `_substitute_var_refs_candidates`'s own + `_VAR_REF_FULL_RE` does not match bash array-subscript syntax + (`${paths[@]}`, `${paths[0]}`) at all (issue #1375's own Fact 5 cites + this exact limitation, and it is the same gap this module's own + `array-literal-assignment-indirection` `KNOWN_BYPASS_COMMANDS` entry + documents for verb reconstruction) -- with no `$NAME`-shaped match + found inside the token, the function harmlessly returns the token's + own text UNCHANGED as if it were already a resolved literal, so + `paths=(a.py b.py); git checkout -- "${paths[@]}"` would otherwise + silently pass the literal string `${paths[@]}` through as a + "resolved" candidate instead of being recognized as still-unresolved. + Confirmed live during this function's own development (before this + check was added).""" + paths: list[str] = [] + for tok in tokens: + if not _is_dynamic(tok): + paths.append(tok) + continue + candidates = _substitute_var_refs_candidates(tok, name_to_raw_value, name_to_raw_value) + if not candidates or any(_is_dynamic(candidate) for candidate in candidates): + return ( + f"a git checkout/restore command has a dynamic path argument ({tok!r}) that could not be " + "resolved to a literal value -- an unresolved path cannot be safely checked against the " + "working tree, so this is denied outright", + (), + ) + paths.extend(candidates) + return None, tuple(paths) + + +def _git_checkout_paths( + tokens_after: list[str], name_to_raw_value: dict[str, str] +) -> tuple[str | None, tuple[str, ...]]: + """checkout_restore_paths for a `git checkout` invocation, TOKENS_AFTER + being every segment token following the literal `checkout` word. + Three sound sub-cases, none requiring a live ref-existence lookup + (issue #1375's own Fact 5 documents the live-git verification each + relies on, git 2.43.0): + + (a) `--` present -- every token after it is a path (git's own + pathspec-disambiguation syntax, and the near-miss's own exact + shape). `git checkout --` with nothing following denies outright: + real git treats this as a harmless no-op, but a downstream pipe or + loop (`git checkout -- | xargs ...`-shaped) could still append + paths at runtime this classifier cannot see, and denying a + genuine no-op costs nothing. + (b) No `--`, 2+ non-flag-shaped positional tokens -- confirmed live + that `git checkout no-such-ref no-such-file` (two unresolvable + positionals, no `--`) reports a pathspec error for BOTH, meaning + whenever real git is given 2+ positionals with no `--`, every + position past the first is a pathspec under every resolution git + can take. Over-including a token that also happens to be a valid + ref name just checks a path that likely does not exist, which is + harmless. + (c) No `--`, exactly one positional token, and it is the literal `.` + or `..` -- both are syntactically invalid git ref names (confirmed + live: `git check-ref-format --branch .`/`--branch ..` both fail, + "not a valid branch name"), so this is unambiguously a path, not a + ref, with no live lookup needed. `git checkout .` on a dirty + tracked file was confirmed live to silently discard the change. + + Bare `git checkout SOMENAME` (single positional, not `.`/`..`, no + `--`) is a deliberate Non-goal: SOMENAME might be a branch/ref name or + a path, and disambiguating soundly needs a live ref-existence lookup + this pure classifier does not perform.""" + if "--" in tokens_after: + after = tokens_after[tokens_after.index("--") + 1 :] + if not after: + return ( + "a 'git checkout --' with no paths following it in this command -- a downstream pipe or loop " + "could append paths at runtime this classifier cannot see, so this is denied outright", + (), + ) + return _resolve_path_tokens(after, name_to_raw_value) + positionals = [t for t in tokens_after if not t.startswith("-")] + if len(positionals) >= 2: + return _resolve_path_tokens(positionals, name_to_raw_value) + if len(positionals) == 1 and not _is_dynamic(positionals[0]) and positionals[0] in (".", ".."): + return _resolve_path_tokens(positionals, name_to_raw_value) + return None, () + + +def _git_restore_paths( + tokens_after: list[str], name_to_raw_value: dict[str, str] +) -> tuple[str | None, tuple[str, ...]]: + """checkout_restore_paths for a `git restore` invocation, TOKENS_AFTER + being every segment token following the literal `restore` word. + Case-sensitive flag walk over an explicit, enumerated vocabulary -- + deliberately NOT reusing `_is_git_push_segment`'s own lower-casing + step, which would collapse `-S` (`--staged`, boolean) and `-s` + (`--source`, value-taking) into the same token and misread a + working-tree-destroying `git restore -s main file.py` as + staged-only-safe (issue #1375's own Fact 5). `saw_staged`/ + `saw_worktree` are last-occurrence-wins (`--staged --no-staged` ends + with `saw_staged=False`); this invocation is safe (empty + checkout_restore_paths, never live-checked) iff `saw_staged` and not + `saw_worktree`. Any flag-shaped token not in this vocabulary -- + including `--pathspec-from-file`/`--pathspec-file-nul`, whose paths + come from a file this classifier cannot inspect -- denies outright + rather than risk under-extracting paths past a flag whose own + value-consumption behavior is unknown here.""" + saw_staged = False + saw_worktree = False + path_tokens: list[str] = [] + i = 0 + n = len(tokens_after) + while i < n: + tok = tokens_after[i] + if tok == "--pathspec-from-file" or tok.startswith("--pathspec-from-file=") or tok == "--pathspec-file-nul": + return ( + "a 'git restore --pathspec-from-file'/'--pathspec-file-nul' flag reads paths from a file this " + "classifier cannot inspect, so this is denied outright", + (), + ) + if tok in ("--staged", "-S"): + saw_staged = True + i += 1 + continue + if tok == "--no-staged": + saw_staged = False + i += 1 + continue + if tok in ("--worktree", "-W"): + saw_worktree = True + i += 1 + continue + if tok == "--no-worktree": + saw_worktree = False + i += 1 + continue + if tok == "--recurse-submodules" or tok.startswith("--recurse-submodules="): + i += 1 + continue + if tok in _RESTORE_BOOLEAN_FLAGS: + i += 1 + continue + if tok in _RESTORE_VALUE_FLAGS: + i += 2 + continue + if tok.startswith("-"): + return ( + f"an unrecognized 'git restore' flag ({tok!r}) -- this classifier cannot safely guarantee " + "correct path extraction past an unrecognized flag that might itself consume the next token, " + "so this is denied outright", + (), + ) + path_tokens.append(tok) + i += 1 + if saw_staged and not saw_worktree: + return None, () + return _resolve_path_tokens(path_tokens, name_to_raw_value) + + +def _find_git_checkout_restore(seg: list[str]) -> tuple[str | None, list[str], bool]: + """Scan SEG (already assignment-stripped, see `_strip_leading_ + assignments`) for a `git checkout`/`git restore` invocation, skipping + past git's own global value-taking options the same way + `_is_git_push_segment` skips past them to find `push` -- but + CASE-SENSITIVELY for the `-C`/`-c` distinction (issue #1375's own Fact + 5: only uppercase `-C`, not lowercase `-c`, relocates which working + tree git operates against; `-c` only sets a config value). + + Scans for a literal `git` token at ANY position in SEG, not just + `seg[0]` -- like `_is_git_push_segment`'s own scan, not anchored to + position 0. A `for VAR in ...; do ...; done` loop is one, real, + non-honest-accident-shaped reason this matters: bash's `for`/`do`/ + `done`/`in` keywords are not shell control operators, so + `segment_tokens` never splits a segment at them, and `git checkout -- + "$f"` sitting after a literal `do` would never be found at `seg[0]`. + Confirmed live during this function's own development that a + seg[0]-anchored version of this scan let `for f in $(git diff + --name-only); do git checkout -- "$f"; done` (this module's own + Acceptance Criteria Map names this exact shape) through with an empty + `checkout_restore_paths` instead of denying on the unresolvable `$f`. + + Returns `(subcommand, tokens_after_subcommand, + saw_tree_relocation_flag)`; `subcommand` is `None` when SEG has no + checkout/restore invocation at all (including when a dynamic token + sits in a position that could be either a global flag/value or the + subcommand itself, immediately after a literal `git` -- a genuinely + ambiguous, non-honest-accident shape this pure classifier declines to + resolve at that specific `git` occurrence, the same disclosed-residual + convention this module's own `KNOWN_BYPASS_COMMANDS` test list already + uses for the analogous dynamic-tool/dynamic-verb case; scanning + continues past it to any later `git` occurrence in the same segment), + in which case the other two return values are meaningless.""" + n = len(seg) + for i, tok in enumerate(seg): + if _is_dynamic(tok) or tok.lower() != "git": + continue + saw_tree_relocation = False + j = i + 1 + ambiguous = False + while j < n: + candidate = seg[j] + if _is_dynamic(candidate): + ambiguous = True + break + if ( + candidate == "-C" + or candidate in _GIT_TREE_RELOCATION_LONG_FLAGS + or any(candidate.startswith(f"{flag}=") for flag in _GIT_TREE_RELOCATION_LONG_FLAGS) + ): + saw_tree_relocation = True + if not candidate.startswith("-"): + break + flag_bare = candidate.split("=", 1)[0] + j += 1 + if "=" not in candidate and ( + flag_bare in _GIT_GLOBAL_SHORT_VALUE_FLAGS or flag_bare in _GIT_LONG_VALUE_FLAGS + ): + j += 1 + if ambiguous: + continue + if j < n and seg[j] in ("checkout", "restore"): + return seg[j], seg[j + 1 :], saw_tree_relocation + return None, [], False + + +def _rule_git_checkout_restore( + segments: list[list[str]], raw_assigned: dict[str, str] +) -> tuple[str | None, tuple[str, ...]]: + """Extract every `checkout_restore_paths` candidate across every + segment of one command, denying outright on any segment where this + classifier cannot soundly determine which working tree is at risk: a + `-C`/`--git-dir`/`--work-tree` global flag on the checkout/restore + segment itself, a `GIT_DIR=`/`GIT_WORK_TREE=`/`GIT_INDEX_FILE=` + assignment anywhere in the command, or a literal `cd` in an earlier + segment of the same command. hooks/check-bash-safety.sh's own new + wrapper step always checks a path against `.cwd` from the PreToolUse + payload (issue #1375's own Fact 5, the cwd-mismatch finding) -- any of + these makes that single, fixed `.cwd` reference point unsound for this + particular invocation, so this denies here (I/O-free -- a token-shape + fact, not a live check) rather than letting the wrapper check the + wrong tree.""" + saw_cd = False + all_paths: list[str] = [] + for seg in segments: + subcommand, tokens_after, saw_tree_relocation = _find_git_checkout_restore(seg) + if subcommand is None: + if any(not _is_dynamic(t) and t == "cd" for t in seg): + saw_cd = True + continue + if saw_tree_relocation or saw_cd or any(name in raw_assigned for name in _GIT_TREE_ENV_VARS): + return ( + f"a 'git {subcommand}' command carries a -C/--git-dir/--work-tree flag, a GIT_DIR=/" + "GIT_WORK_TREE=/GIT_INDEX_FILE= assignment, or an earlier 'cd' in the same command -- this " + "classifier cannot soundly determine which working tree is at risk, so this is denied outright", + (), + ) + if subcommand == "checkout": + deny_reason, paths = _git_checkout_paths(tokens_after, raw_assigned) + else: + deny_reason, paths = _git_restore_paths(tokens_after, raw_assigned) + if deny_reason: + return deny_reason, () + all_paths.extend(paths) + return None, tuple(all_paths) + + def _resolve_seg_tokens_candidates( tokens: list[str], name_to_value: dict[str, str], name_to_raw_value: dict[str, str] ) -> set[str] | None: @@ -2702,18 +3038,19 @@ def _classify_tokens( outer_literals = outer_name_to_value or {} outer_raw = outer_name_to_raw_value or {} - content_reason, content_is_git_push = _rule_command_substitution_content(tokens) + content_reason, content_is_git_push, content_checkout_restore_paths = _rule_command_substitution_content(tokens) if content_reason: - return Verdict(True, content_reason, content_is_git_push) + return Verdict(True, content_reason, content_is_git_push, content_checkout_restore_paths) - array_content_reason, array_content_is_git_push = _rule_array_literal_content( + array_content_reason, array_content_is_git_push, array_content_checkout_restore_paths = _rule_array_literal_content( tokens, {**outer_literals, **_assigned_literals(tokens)}, {**outer_raw, **_assigned_raw_values(tokens)}, ) is_git_push = content_is_git_push or array_content_is_git_push + checkout_restore_paths = content_checkout_restore_paths + array_content_checkout_restore_paths if array_content_reason: - return Verdict(True, array_content_reason, is_git_push) + return Verdict(True, array_content_reason, is_git_push, checkout_restore_paths) tokens = _fold_array_literal_spans(_fold_command_substitution_spans(tokens)) segments = [s for s in (_strip_leading_assignments(seg) for seg in segment_tokens(tokens)) if s] @@ -2725,16 +3062,16 @@ def _classify_tokens( literal_hit = _rule_a_literal(segments) if literal_hit: - return Verdict(True, literal_hit, is_git_push) + return Verdict(True, literal_hit, is_git_push, checkout_restore_paths) gh_api_hit = _rule_gh_api_write(segments, lowered_command, assigned, raw_assigned) if gh_api_hit: - return Verdict(True, gh_api_hit, is_git_push) + return Verdict(True, gh_api_hit, is_git_push, checkout_restore_paths) loop_hit, loop_is_git_push = _segment_loop_hit(segments, assigned, raw_assigned) is_git_push = is_git_push or loop_is_git_push if loop_hit: - return Verdict(True, loop_hit, is_git_push) + return Verdict(True, loop_hit, is_git_push, checkout_restore_paths) collapsed_segments = [ collapsed for seg in segments if (collapsed := _strip_leading_unassigned_bare_refs(seg, raw_assigned)) @@ -2743,9 +3080,19 @@ def _classify_tokens( collapsed_hit, collapsed_is_git_push = _segment_loop_hit(collapsed_segments, assigned, raw_assigned) is_git_push = is_git_push or collapsed_is_git_push if collapsed_hit: - return Verdict(True, f"{collapsed_hit}, once a leading unassigned reference word-split away", is_git_push) + return Verdict( + True, + f"{collapsed_hit}, once a leading unassigned reference word-split away", + is_git_push, + checkout_restore_paths, + ) + + own_checkout_restore_hit, own_checkout_restore_paths = _rule_git_checkout_restore(segments, raw_assigned) + checkout_restore_paths = checkout_restore_paths + own_checkout_restore_paths + if own_checkout_restore_hit: + return Verdict(True, own_checkout_restore_hit, is_git_push, checkout_restore_paths) - return Verdict(False, "no denied pattern matched", is_git_push) + return Verdict(False, "no denied pattern matched", is_git_push, checkout_restore_paths) # --- stdin JSON entrypoint ------------------------------------------------ @@ -2795,6 +3142,12 @@ def main() -> int: "decision": "deny" if verdict.deny else "allow", "reason": verdict.reason, "is_git_push": verdict.is_git_push, + # Issue #1375: a genuine JSON array, not a newline-joined + # string -- a path containing a newline would otherwise + # split into fragments that each match nothing on the live + # `git diff` check and silently pass. `json.dumps` encodes + # a tuple as a JSON array natively. + "checkout_restore_paths": verdict.checkout_restore_paths, } ) ) diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 3190e439..4516aa27 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -1357,7 +1357,7 @@ def test_rule_command_substitution_content_detects_an_embedded_install(tool: str a punctuation character shlex breaks a word at, so an assignment's `NAME=` prefix stays fused onto the leading `$` in the same token.""" tokens = ["x=$", "(", tool, "install", "evil-pkg", ")"] - reason, _ = checker._rule_command_substitution_content(tokens) + reason, _, _ = checker._rule_command_substitution_content(tokens) assert reason is not None @@ -1373,7 +1373,7 @@ def test_rule_command_substitution_content_allows_harmless_inner_content(value: silently dropping a non-denying inner `is_git_push=True` signal (see the function's own docstring).""" tokens = ["echo", "$", "(", "date", value, ")"] - assert checker._rule_command_substitution_content(tokens) == (None, False) + assert checker._rule_command_substitution_content(tokens) == (None, False, ()) # --- Issue #1326 Stage 1, fifteenth round: bash's own leading-assignment ---- @@ -1492,7 +1492,7 @@ def test_rule_array_literal_content_detects_a_denied_pair_regardless_of_a_leadin `Y=1; A=(uv install $Y); "${A[@]}"` was wrongly ALLOWED before this function existed.""" tokens = ["dummy=", "(", f"${first}", "uv", "install", f"${second}", ")"] - reason, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) assert reason is not None @@ -1511,7 +1511,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_bare_ref(unse fused with other text (not a bare whole-token reference), must NOT be collapsed -- that shape does not word-split away to nothing.""" tokens = ["dummy=", "(", f"${unset_name}", verb_a, "install", ")"] - reason, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) assert reason is not None @@ -1531,13 +1531,13 @@ def test_rule_array_literal_content_allows_harmless_content() -> None: denied pattern, with or without a leading unassigned reference, stays allowed.""" tokens = ["dummy=", "(", "$NEVERSET", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False) + assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False, ()) def test_rule_array_literal_content_no_span_present() -> None: """Robustness: a token stream with no array-literal span at all (e.g. an ordinary command) returns cleanly, never a crash.""" - assert checker._rule_array_literal_content(["echo", "hi"], {}, {}) == (None, False) + assert checker._rule_array_literal_content(["echo", "hi"], {}, {}) == (None, False, ()) def test_strip_leading_unassigned_bare_refs_stops_at_a_fused_token() -> None: @@ -1563,7 +1563,7 @@ def test_rule_array_literal_content_empty_array_is_harmless() -> None: """No false positive / no crash: an empty array literal `NAME=()` has no inner content to recursively classify at all.""" tokens = ["dummy=", "(", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False) + assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False, ()) def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leading_unassigned_ref() -> None: @@ -1572,7 +1572,7 @@ def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leadin `_strip_leading_unassigned_bare_refs` to strip -- the collapsed reading equals the as-is one, so only one classification is needed.""" tokens = ["dummy=", "(", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False) + assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False, ()) def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> None: @@ -1589,7 +1589,7 @@ def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> No real, with a dynamic verb argument right after it -- exactly B2's own watched shape.""" tokens = ["dummy=", "(", "$NEVERSET", "uv", "$VERB", ")"] - reason, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) assert reason is not None assert "unassigned reference" in reason @@ -1631,7 +1631,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_braced_bare_r this round, silently degrading the collapsed reading to a no-op for this shape.""" tokens = ["dummy=", "(", f"${{{unset_name}}}", verb_a, "install", ")"] - reason, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) assert reason is not None @@ -1646,7 +1646,7 @@ def test_rule_array_literal_content_detects_an_outer_scope_resolved_pair() -> No recursive `_classify_tokens` call.""" tokens = ["dummy=", "(", "$G", "$P", "$M", ")"] outer = {"G": "gh", "P": "pr", "M": "merge"} - reason, _ = checker._rule_array_literal_content(tokens, outer, outer) + reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer) assert reason is not None @@ -1743,7 +1743,7 @@ def test_rule_array_literal_content_detects_a_braced_subscript_decoy() -> None: the subscript decoy blocked it from ever firing until it collapsed away.""" tokens = ["dummy=", "(", "${NEVERSET[0]}", "uv", "$VERB", ")"] - reason, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) assert reason is not None @@ -1754,7 +1754,7 @@ def test_rule_array_literal_content_detects_a_fused_reference_chain_decoy() -> N before a fused chain of two bare references was recognized as vanishing as a unit.""" tokens = ["dummy=", "(", "$A_UNSET$B_UNSET", "gh", "pr", "merge", "1", ")"] - reason, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) assert reason is not None @@ -1887,7 +1887,7 @@ def test_rule_command_substitution_content_scans_second_fused_span_in_same_token this test only proves that fix reached end-to-end through `_rule_command_substitution_content`'s own scan loop.""" tokens = ["echo", "$(echo ok)$(pip install evil-pkg)"] - reason, _ = checker._rule_command_substitution_content(tokens) + reason, _, _ = checker._rule_command_substitution_content(tokens) assert reason is not None @@ -1896,20 +1896,20 @@ def test_rule_command_substitution_content_skips_blank_fused_span_then_finds_den skipped without denying by itself, but scanning continues to the next fused span in the same token.""" tokens = ["echo", "$( )$(pip install evil-pkg)"] - reason, _ = checker._rule_command_substitution_content(tokens) + reason, _, _ = checker._rule_command_substitution_content(tokens) assert reason is not None def test_rule_command_substitution_content_both_fused_spans_harmless() -> None: tokens = ["echo", "$(echo ok)$(echo also-ok)"] - assert checker._rule_command_substitution_content(tokens) == (None, False) + assert checker._rule_command_substitution_content(tokens) == (None, False, ()) def test_rule_command_substitution_content_empty_unquoted_span_skipped() -> None: """An empty, unquoted `$()` substitution has no inner tokens to recurse into -- distinct from the fused/quoted empty-span case above.""" tokens = ["$", "(", ")"] - assert checker._rule_command_substitution_content(tokens) == (None, False) + assert checker._rule_command_substitution_content(tokens) == (None, False, ()) def test_tokenize_raises_on_unbalanced_quote() -> None: @@ -2613,3 +2613,490 @@ def test_main_denies_a_real_denied_command(monkeypatch: pytest.MonkeyPatch, caps def test_main_allows_a_harmless_command(monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str]) -> None: payload = {"tool_name": "Bash", "tool_input": {"command": "echo hi"}} assert _run_main(payload, monkeypatch, capsys)["decision"] == "allow" + + +# --- Issue #1375: git checkout/restore path extraction. `classify()` stays +# I/O-free (this module's own established architecture); this section only +# extracts every candidate path a checkout/restore invocation could +# discard, for hooks/check-bash-safety.sh's own new wrapper step to check +# live against the real working tree. See hooks/gitapex_check_bash_safety.py's +# own "git checkout/restore path extraction" section for the full design +# rationale and the live-git verification (git 2.43.0) it is built on. + +_PATH_TOKENS = st.text(alphabet=string.ascii_letters + string.digits + "_./", min_size=1, max_size=12).filter( + lambda p: not p.startswith("-") and p not in (".", "..") +) + + +@_PROPERTIES +@given(paths=st.lists(_PATH_TOKENS, min_size=1, max_size=5)) +def test_resolve_path_tokens_returns_literal_tokens_unchanged(paths: list[str]) -> None: + """Model-based: every literal (non-dynamic) token is returned as-is, in + order, with no deny reason.""" + reason, resolved = checker._resolve_path_tokens(paths, {}) + assert reason is None + assert resolved == tuple(paths) + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_PATH_TOKENS) +def test_resolve_path_tokens_resolves_a_braced_reference_case_preserved(name: str, value: str) -> None: + """Model-based: a dynamic `${NAME}` path token resolves to NAME's own + CASE-PRESERVED raw value -- unlike every other caller of + `_substitute_var_refs_candidates` in this module (which compares a + resolved value case-insensitively against a known tool/verb/flag + literal via the lowercased `name_to_value`), a filesystem path is + case-sensitive, so this must resolve against the raw, case-preserving + map. Regression pin: an earlier version of this function resolved + against the lowercased map and would have silently mismatched a + mixed-case path like `README.md` against `readme.md`.""" + mixed_case_value = value.swapcase() + reason, resolved = checker._resolve_path_tokens([f"${{{name}}}"], {name: mixed_case_value}) + assert reason is None + assert resolved == (mixed_case_value,) + + +@_PROPERTIES +@given(name=_IDENTIFIERS) +def test_resolve_path_tokens_denies_an_unresolvable_dynamic_token(name: str) -> None: + """Fail-closed: a dynamic token referencing a name that is never + assigned cannot be resolved to a literal -- denied outright here + rather than passed to the live wrapper check empty-handed, since + `git diff --quiet HEAD -- PATH` exits 0 (clean) for a path that does + not exist (issue #1375 Fact 5, confirmed live), which would be + fail-open.""" + reason, resolved = checker._resolve_path_tokens([f"${name}"], {}) + assert reason is not None + assert resolved == () + + +def test_resolve_path_tokens_denies_an_array_subscript_token() -> None: + """Regression pin, found during this function's own development: bash + array-subscript syntax (`${paths[@]}`) is not matched by + `_substitute_var_refs_candidates`'s own `_VAR_REF_FULL_RE` at all + (issue #1375 Fact 5 cites this exact limitation), so with no `$NAME` + match found inside the token, that function harmlessly returns the + token's own text UNCHANGED -- silently treating an unexpanded shell + construct as though it were already a resolved literal path. Must + deny, not pass `${paths[@]}` through as a literal filename.""" + reason, resolved = checker._resolve_path_tokens(["${paths[@]}"], {}) + assert reason is not None + assert resolved == () + + +@_PROPERTIES +@given(paths=st.lists(_PATH_TOKENS, min_size=1, max_size=4)) +def test_git_checkout_paths_extracts_every_token_after_double_dash(paths: list[str]) -> None: + """Model-based, sub-case (a): every token after a literal `--` is a + path -- the near-miss's own exact shape (`git checkout -- PATH`).""" + reason, resolved = checker._git_checkout_paths(["--", *paths], {}) + assert reason is None + assert resolved == tuple(paths) + + +def test_git_checkout_paths_denies_double_dash_with_nothing_following() -> None: + """`git checkout --` with no paths following denies outright: a + harmless no-op in real git by itself, but a downstream pipe/loop could + still append paths at runtime this classifier cannot see, and denying + a genuine no-op costs nothing.""" + reason, resolved = checker._git_checkout_paths(["--"], {}) + assert reason is not None + assert resolved == () + + +@_PROPERTIES +@given(paths=st.lists(_PATH_TOKENS, min_size=2, max_size=4)) +def test_git_checkout_paths_extracts_two_or_more_positionals_with_no_double_dash(paths: list[str]) -> None: + """Model-based, sub-case (b): with no `--`, 2+ non-flag-shaped + positionals are ALL read as paths -- confirmed live that + `git checkout no-such-ref no-such-file` reports a pathspec error for + BOTH arguments, so every position past the first is a pathspec under + every resolution real git can take once one exists at all.""" + reason, resolved = checker._git_checkout_paths(paths, {}) + assert reason is None + assert resolved == tuple(paths) + + +@_PROPERTIES +@given(dot=st.sampled_from([".", ".."])) +def test_git_checkout_paths_treats_a_single_dot_or_dotdot_positional_as_a_path(dot: str) -> None: + """Model-based, sub-case (c): a lone `.`/`..` positional (no `--`) is + a path, not a ref -- both are syntactically invalid git ref names + (confirmed live: `git check-ref-format --branch .`/`--branch ..` both + fail), and `git checkout .` on a dirty tracked file was confirmed live + to silently discard the change.""" + reason, resolved = checker._git_checkout_paths([dot], {}) + assert reason is None + assert resolved == (dot,) + + +@_PROPERTIES +@given(name=_PATH_TOKENS) +def test_git_checkout_paths_is_a_non_goal_for_a_single_bare_positional(name: str) -> None: + """No false positive: a single positional that is not `.`/`..` (e.g. + a branch name) is a deliberate Non-goal -- disambiguating a bare + `git checkout SOMENAME` from a branch/ref name needs a live + ref-existence lookup this pure classifier does not perform.""" + assume(name not in (".", "..")) + reason, resolved = checker._git_checkout_paths([name], {}) + assert reason is None + assert resolved == () + + +def test_git_checkout_paths_allows_a_flag_only_invocation() -> None: + """No false positive: `git checkout -b new-branch` has one + flag-shaped and one non-flag-shaped token, but the non-flag token is + a branch name, not `.`/`..` -- stays the Non-goal, empty paths.""" + reason, resolved = checker._git_checkout_paths(["-b", "new-branch"], {}) + assert reason is None + assert resolved == () + + +@_PROPERTIES +@given(staged=st.sampled_from(["--staged", "-S"]), paths=st.lists(_PATH_TOKENS, min_size=0, max_size=3)) +def test_git_restore_paths_empty_when_staged_without_worktree(staged: str, paths: list[str]) -> None: + """Model-based: `--staged`/`-S` without `--worktree` never touches the + working tree -- empty `checkout_restore_paths`, never live-checked, + regardless of what path arguments are also present.""" + reason, resolved = checker._git_restore_paths([staged, *paths], {}) + assert reason is None + assert resolved == () + + +@_PROPERTIES +@given(worktree=st.sampled_from(["--worktree", "-W"]), paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) +def test_git_restore_paths_checked_when_staged_and_worktree_both_present(worktree: str, paths: list[str]) -> None: + """Model-based, regression pin for issue #1375's own Fact 5: `--staged + --worktree PATH` is a real working-tree-affecting restore despite + `--staged` being present -- `saw_worktree=True` must still force the + path to be checked.""" + reason, resolved = checker._git_restore_paths(["--staged", worktree, *paths], {}) + assert reason is None + assert resolved == tuple(paths) + + +@_PROPERTIES +@given(paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) +def test_git_restore_paths_checked_with_no_flags_at_all(paths: list[str]) -> None: + """Model-based: a bare `git restore PATH` with no flags at all is + never staged-only-safe -- always checked.""" + reason, resolved = checker._git_restore_paths(paths, {}) + assert reason is None + assert resolved == tuple(paths) + + +@_PROPERTIES +@given(ref=_PATH_TOKENS, paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) +def test_git_restore_paths_checked_for_source_short_flag_not_conflated_with_staged(ref: str, paths: list[str]) -> None: + """Model-based, regression pin for issue #1375's own Fact 5: `-s` + (`--source`, value-taking) must never be conflated with `-S` + (`--staged`, boolean) the way a lower-casing flag scan (like + `_is_git_push_segment`'s own) would -- `git restore -s main PATH` + stays checked, not wrongly read as staged-only-safe.""" + reason, resolved = checker._git_restore_paths(["-s", ref, *paths], {}) + assert reason is None + assert resolved == tuple(paths) + + +@_PROPERTIES +@given(last=st.sampled_from(["--staged", "--no-staged"])) +def test_git_restore_paths_last_occurrence_wins_for_staged(last: str) -> None: + """Model-based: `saw_staged` is last-occurrence-wins -- `--staged + --no-staged` ends with `saw_staged=False` (checked), and `--no-staged + --staged` ends with `saw_staged=True` (empty, iff no `--worktree`).""" + flags = ["--no-staged", "--staged"] if last == "--staged" else ["--staged", "--no-staged"] + reason, resolved = checker._git_restore_paths([*flags, "f.py"], {}) + assert reason is None + if last == "--staged": + assert resolved == () + else: + assert resolved == ("f.py",) + + +@_PROPERTIES +@given(flag=st.sampled_from(["--pathspec-from-file=list.txt", "--pathspec-from-file", "--pathspec-file-nul"])) +def test_git_restore_paths_denies_pathspec_from_file(flag: str) -> None: + """Paths sourced from a file this classifier cannot inspect deny + outright rather than silently under-extracting (an empty + `checkout_restore_paths` would be exactly issue #1375 Fact 5's own + fail-open shape).""" + reason, resolved = checker._git_restore_paths([flag], {}) + assert reason is not None + assert resolved == () + + +@_PROPERTIES +@given(flag=st.text(alphabet=string.ascii_lowercase, min_size=1, max_size=8).map(lambda s: f"--{s}")) +def test_git_restore_paths_denies_an_unrecognized_flag(flag: str) -> None: + """Fail-closed: any flag-shaped token outside the enumerated + vocabulary denies outright -- this classifier cannot safely guarantee + correct path extraction past a flag whose own value-consumption + behavior it does not know.""" + assume(flag not in checker._RESTORE_BOOLEAN_FLAGS | checker._RESTORE_VALUE_FLAGS) + assume(not flag.startswith("--pathspec-from-file") and not flag.startswith("--recurse-submodules")) + assume(flag not in ("--staged", "--no-staged", "--worktree", "--no-worktree")) + reason, resolved = checker._git_restore_paths([flag], {}) + assert reason is not None + assert resolved == () + + +@_PROPERTIES +@given(prefix=st.lists(_PATH_TOKENS, min_size=0, max_size=3), paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) +def test_find_git_checkout_restore_finds_git_at_any_segment_position(prefix: list[str], paths: list[str]) -> None: + """Model-based, regression pin found during this function's own + development: scans for a literal `git` token at ANY position in the + segment, not just `seg[0]` -- like `_is_git_push_segment`'s own scan. + A `for VAR in ...; do ...; done` loop is one real reason this matters: + bash's `for`/`do`/`done`/`in` keywords are not shell control + operators, so `segment_tokens` never splits a segment at them, and + `git checkout -- PATH` sitting after a literal `do` would never be + found at `seg[0]`.""" + assume(all(p != "git" for p in prefix)) + seg = [*prefix, "git", "checkout", "--", *paths] + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg) + assert subcommand == "checkout" + assert tokens_after == ["--", *paths] + assert saw_tree_relocation is False + + +def test_find_git_checkout_restore_none_for_a_segment_with_no_git() -> None: + """No false positive: a segment with no literal `git` token at all is + never treated as a checkout/restore invocation.""" + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["echo", "checkout", "restore"]) + assert subcommand is None + + +@_PROPERTIES +@given(flag=st.sampled_from(["-C", "--git-dir", "--work-tree"])) +def test_find_git_checkout_restore_flags_tree_relocation(flag: str) -> None: + """Model-based: `-C`/`--git-dir`/`--work-tree` (global flags that + relocate which working tree git operates against) are flagged + regardless of their own value -- the caller uses this to deny outright + rather than let the live wrapper check the wrong tree (issue #1375's + own Fact 5 cwd finding).""" + seg = ["git", flag, "/some/path", "checkout", "--", "f.py"] + subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg) + assert subcommand == "checkout" + assert saw_tree_relocation is True + + +def test_find_git_checkout_restore_does_not_flag_lowercase_c_config_flag() -> None: + """No false positive: `-c` (lowercase, sets a config value) is + case-sensitively distinct from `-C` (uppercase, relocates the working + tree) and must never be conflated with it (issue #1375's own Fact 5).""" + seg = ["git", "-c", "user.name=x", "checkout", "--", "f.py"] + subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg) + assert subcommand == "checkout" + assert saw_tree_relocation is False + + +def test_find_git_checkout_restore_is_a_non_goal_for_a_dynamic_subcommand() -> None: + """No false positive (disclosed Non-goal): a dynamically constructed + subcommand name (`V=checkout; git $V -- f.py`) is not honest-accident- + shaped and is not detected -- the same disclosed-residual convention + this module's own `KNOWN_BYPASS_COMMANDS` test list already uses for + the analogous dynamic-tool/dynamic-verb case.""" + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["git", "$V", "--", "f.py"]) + assert subcommand is None + + +@_PROPERTIES +@given(command_paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) +def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_paths: list[str]) -> None: + """Model-based: multiple checkout/restore invocations chained in one + command (`git checkout -- a.py; git restore b.py`) accumulate paths + from every segment, not just the first.""" + segments = [["git", "checkout", "--", *command_paths], ["git", "restore", *command_paths]] + reason, resolved = checker._rule_git_checkout_restore(segments, {}) + assert reason is None + assert resolved == (*command_paths, *command_paths) + + +def test_rule_git_checkout_restore_denies_when_git_dir_env_var_assigned() -> None: + """Model-based, regression pin for issue #1375's own Fact 5: a + `GIT_DIR=`/`GIT_WORK_TREE=`/`GIT_INDEX_FILE=` assignment anywhere in + the command makes the wrapper's own fixed `.cwd` reference point + unsound -- denied outright by the classifier itself (I/O-free, a + token-shape fact) rather than letting the live wrapper check the + wrong tree.""" + segments = [["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}) + assert reason is not None + assert resolved == () + + +def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_cd() -> None: + """Model-based, regression pin for issue #1375's own Fact 5: an + earlier segment in the same command containing a literal `cd` makes + the wrapper's own fixed `.cwd` unsound for a LATER checkout/restore + segment -- denied outright.""" + segments = [["cd", "/tmp"], ["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {}) + assert reason is not None + assert resolved == () + + +def test_rule_git_checkout_restore_allows_cd_after_the_checkout_segment() -> None: + """No false positive: `_rule_git_checkout_restore` only denies for a + `cd` in an EARLIER segment -- a `cd` AFTER the checkout/restore segment + does not retroactively make the already-scanned segment unsound.""" + segments = [["git", "checkout", "--", "f.py"], ["cd", "/tmp"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {}) + assert reason is None + assert resolved == ("f.py",) + + +# --- End-to-end classify() coverage, pinning every explicit safe/deny case +# issue #1375's own Acceptance Criteria Map and "Explicit safe cases" +# section name by hand. + + +@_PROPERTIES +@given(ref=st.sampled_from(["main", "HEAD~1", "some-branch"])) +def test_classify_allows_ordinary_branch_switching(ref: str) -> None: + verdict = checker.classify(f"git checkout {ref}") + assert verdict.deny is False + assert verdict.checkout_restore_paths == () + + +def test_classify_allows_checkout_dash_b() -> None: + verdict = checker.classify("git checkout -b new-branch") + assert verdict.deny is False + assert verdict.checkout_restore_paths == () + + +def test_classify_allows_restore_staged_only() -> None: + verdict = checker.classify("git restore --staged f.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == () + + +def test_classify_extracts_path_for_checkout_double_dash() -> None: + verdict = checker.classify("git checkout -- f.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("f.py",) + + +def test_classify_extracts_dot_for_checkout_dot() -> None: + verdict = checker.classify("git checkout .") + assert verdict.deny is False + assert verdict.checkout_restore_paths == (".",) + + +def test_classify_extracts_path_for_bare_restore() -> None: + verdict = checker.classify("git restore f.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("f.py",) + + +def test_classify_resolves_a_same_command_assignment_to_a_literal_path() -> None: + """A dynamic path token that resolves to a literal via a same-command + assignment (`f=README.md; git checkout -- "$f"`) is substituted and + surfaced as a candidate, not denied.""" + verdict = checker.classify('f=README.md; git checkout -- "$f"') + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("README.md",) + + +def test_classify_denies_a_loop_fed_dynamic_checkout_path() -> None: + """Regression pin for issue #1375's own Acceptance Criteria Map: a + `for`-loop-fed dynamic path with no same-command assignment for the + loop variable is unresolvable and denies outright, rather than + silently passing through with an empty `checkout_restore_paths`.""" + verdict = checker.classify('for f in $(git diff --name-only); do git checkout -- "$f"; done') + assert verdict.deny is True + + +def test_classify_denies_an_array_subscript_fed_checkout_path() -> None: + """Regression pin for issue #1375's own Acceptance Criteria Map: the + `${paths[@]}`-shaped array-subscript indirection is a real, pinned + `KNOWN_BYPASS` shape (this module's own `array-literal-assignment- + indirection` entry) for the same underlying `_VAR_REF_FULL_RE` + limitation -- must deny, not silently pass an unresolved literal + string through as a path.""" + verdict = checker.classify('paths=(a.py b.py); git checkout -- "${paths[@]}"') + assert verdict.deny is True + + +def test_classify_threads_checkout_restore_paths_through_command_substitution() -> None: + """Regression pin, mirroring the fifteenth-round `is_git_push` + recursion-drop fix this module already carries: a checkout/restore + invocation embedded in a `$(...)` command substitution must still + surface its own `checkout_restore_paths` in the outer `Verdict`, not + silently drop it the way an earlier version of this function would + have.""" + verdict = checker.classify("x=$(git checkout -- f.py)") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("f.py",) + + +def test_classify_threads_checkout_restore_paths_through_array_literal() -> None: + """Regression pin, mirroring `_rule_array_literal_content`'s own + nineteenth-round outer-scope fix: a checkout/restore invocation + embedded in a `NAME=(...)` array literal must still surface its own + `checkout_restore_paths` in the outer `Verdict`.""" + verdict = checker.classify('A=(git checkout -- f.py); "${A[@]}"') + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("f.py",) + + +def test_classify_denies_checkout_with_a_tree_relocation_flag() -> None: + """Regression pin for issue #1375's own Fact 5 cwd finding: a `-C` + global flag makes the wrapper's fixed `.cwd` unsound for this + invocation -- denied outright by the classifier.""" + verdict = checker.classify("git -C /tmp/some-repo checkout -- f.py") + assert verdict.deny is True + + +def test_classify_denies_checkout_after_an_earlier_cd() -> None: + """Regression pin for issue #1375's own Fact 5 cwd finding: an + earlier `cd` in the same command makes the wrapper's fixed `.cwd` + unsound for a later checkout -- denied outright.""" + verdict = checker.classify("cd /tmp; git checkout -- f.py") + assert verdict.deny is True + + +def test_classify_does_not_flag_checkout_restore_prose_inside_a_commit_message() -> None: + """No false positive: this module's own established convention (see + its module docstring's own "no substring/prose fallback" constraint) + -- `git checkout`-shaped TEXT sitting inside an unrelated command's own + quoted string argument is not a real invocation and must never be + flagged, unlike `_rule_a_literal`'s own deliberate same-token + literal-phrase fallback for install verbs (a `deny`-severity false + positive here would fire exactly when files are legitimately dirty, + which this gate cannot tolerate the way a `warn` could).""" + verdict = checker.classify('git commit -m "revert via git checkout -- foo.py"') + assert verdict.deny is False + assert verdict.checkout_restore_paths == () + + +def test_classify_leaves_ordinary_git_push_unaffected() -> None: + """No regression: this is purely additive detection surface -- an + ordinary `git push` must still classify exactly as before, with no + checkout_restore_paths.""" + verdict = checker.classify("git push origin main") + assert verdict.deny is False + assert verdict.is_git_push is True + assert verdict.checkout_restore_paths == () + + +def test_main_output_includes_checkout_restore_paths_for_a_checkout_command( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + """End-to-end: the stdin-JSON entrypoint surfaces `checkout_restore_ + paths` as a genuine JSON array (issue #1375: not a newline-joined + string, so hooks/check-bash-safety.sh's own new wrapper step can + base64-decode each element safely even if a path contains a + newline).""" + payload = {"tool_name": "Bash", "tool_input": {"command": "git checkout -- f.py"}} + out = _run_main(payload, monkeypatch, capsys) + assert out["decision"] == "allow" + assert out["checkout_restore_paths"] == ["f.py"] + + +def test_main_output_includes_empty_checkout_restore_paths_for_a_harmless_command( + monkeypatch: pytest.MonkeyPatch, capsys: pytest.CaptureFixture[str] +) -> None: + payload = {"tool_name": "Bash", "tool_input": {"command": "echo hi"}} + out = _run_main(payload, monkeypatch, capsys) + assert out["checkout_restore_paths"] == [] From 32c5e459cbba56301e3a492ba427f288068dc598 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 09:52:34 +0000 Subject: [PATCH 02/46] test(hooks): add end-to-end wrapper regression tests for checkout/restore Covers the new git-checkout/restore wrapper step in check-bash-safety.sh against real scratch git repos: a dirty target denies from both the repo root and a subdirectory, a clean target and ordinary branch switches allow, restoring with only the staged flag stays allowed regardless of working-tree dirtiness, a missing or non-repo cwd denies, the unborn-HEAD empty-tree fallback behaves correctly in both directions, a tree-location flag denies through the classifier before any live git call runs, and a real merge conflict names a remedy that actually works mid-conflict (unlike a plain stash, which fails while unresolved). Refs #1375. --- hooks/test_gitapex_check_bash_safety.py | 215 +++++++++++++++++++++++- 1 file changed, 213 insertions(+), 2 deletions(-) diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index a702bf40..172e5689 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -55,9 +55,24 @@ def run( - command: str, tool_name: object = "Bash", extra_env: dict[str, str] | None = None + command: str, + tool_name: object = "Bash", + extra_env: dict[str, str] | None = None, + payload_cwd: str | None = None, ) -> subprocess.CompletedProcess[str]: - payload = json.dumps({"tool_name": tool_name, "tool_input": {"command": command}}) + """PAYLOAD_CWD (issue #1375) sets the PreToolUse payload's own `.cwd` + field -- the Bash tool call's own working directory, as Claude Code's + real hook JSON carries it -- kept distinct from this `subprocess.run` + call's own `cwd=` below (always REPO_ROOT, so the script can find its + own classifier companion file via `BASH_SOURCE`), the same split + hooks/check-bash-safety.sh's own new git checkout/restore wrapper step + relies on: it reads `.cwd` from the payload for its live `git diff` + check, never `${CLAUDE_PROJECT_DIR:-$(pwd)}`.""" + tool_input: dict[str, object] = {"command": command} + payload_obj: dict[str, object] = {"tool_name": tool_name, "tool_input": tool_input} + if payload_cwd is not None: + payload_obj["cwd"] = payload_cwd + payload = json.dumps(payload_obj) env = dict(os.environ) env.pop("CLAUDE_PROJECT_DIR", None) if extra_env: @@ -1258,3 +1273,199 @@ def test_git_push_silent_when_scan_finds_nothing(tmp_path: Path) -> None: assert result.returncode == 0 assert result.stdout == "" assert result.stderr == "" + + +# --- Finding 5: git checkout/restore gated on a live git-diff check (issue +# #1375). End-to-end regression suite for hooks/check-bash-safety.sh's own +# new wrapper step, matching this file's own established convention: run +# the shipped script via subprocess against a real scratch git repo, rather +# than re-deriving its behavior in Python. + + +def _git(repo_dir: Path, *args: str) -> subprocess.CompletedProcess[str]: + return subprocess.run( + ["git", "-c", "commit.gpgsign=false", *args], + cwd=str(repo_dir), + check=True, + capture_output=True, + text=True, + timeout=10, + env={**os.environ, "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_NOSYSTEM": "1"}, + ) + + +def _init_repo_with_committed_file(repo_dir: Path, filename: str = "f.py", content: str = "hello\n") -> Path: + repo_dir.mkdir(parents=True, exist_ok=True) + _git(repo_dir, "init", "-q") + # Pinned explicitly rather than relying on the host's own + # `init.defaultBranch` (matches `_init_diverged_repo`'s own established + # convention above, for the identical determinism reason): a test that + # later checks out a branch literally named "main" must not depend on + # what a given git installation happens to default to. + _git(repo_dir, "symbolic-ref", "HEAD", "refs/heads/main") + _git(repo_dir, "config", "user.email", "test@example.com") + _git(repo_dir, "config", "user.name", "Test") + file_path = repo_dir / filename + file_path.write_text(content) + _git(repo_dir, "add", filename) + _git(repo_dir, "commit", "-q", "-m", "base commit") + return file_path + + +def test_checkout_denied_when_target_has_uncommitted_changes(tmp_path: Path) -> None: + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir) + file_path.write_text("hello\ndirty\n") + result = run("git checkout -- f.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert payload["hookSpecificOutput"]["permissionDecision"] == "deny" + assert "f.py" in payload["systemMessage"] + assert "git stash" in payload["systemMessage"] + + +def test_checkout_denied_from_a_subdirectory_when_target_has_uncommitted_changes(tmp_path: Path) -> None: + """The near-miss's own exact shape (issue #1375, issue #1128 repair 4): + replayed from a SUBDIRECTORY of the repo, not just the repo root -- + `.cwd` (Claude Code's own record of the Bash tool call's actual + working directory) must be what the live check resolves the pathspec + against, not this hook runner's own `${CLAUDE_PROJECT_DIR:-$(pwd)}`.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir) + file_path.write_text("hello\ndirty\n") + subdir = repo_dir / "sub" + subdir.mkdir() + result = run("git checkout -- ../f.py", payload_cwd=str(subdir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert payload["hookSpecificOutput"]["permissionDecision"] == "deny" + + +def test_checkout_allowed_when_target_is_clean(tmp_path: Path) -> None: + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("git checkout -- f.py", payload_cwd=str(repo_dir)) + assert result.returncode == 0, f"stderr={result.stderr!r}" + assert result.stdout == "" + assert result.stderr == "" + + +def test_checkout_dot_denied_when_a_tracked_file_is_dirty(tmp_path: Path) -> None: + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir) + file_path.write_text("hello\ndirty\n") + result = run("git checkout .", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_ordinary_branch_switch_allowed(tmp_path: Path) -> None: + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("git checkout -b feature-x", payload_cwd=str(repo_dir)) + assert result.returncode == 0, f"stderr={result.stderr!r}" + assert result.stdout == "" + assert result.stderr == "" + + +def test_restore_denied_when_target_has_uncommitted_changes(tmp_path: Path) -> None: + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir) + file_path.write_text("hello\ndirty\n") + result = run("git restore f.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_restore_staged_allowed_even_when_worktree_is_dirty(tmp_path: Path) -> None: + """`git restore --staged PATH` never touches the working tree -- + `checkout_restore_paths` stays empty for it (never live-checked), so + this must be allowed regardless of the file's own working-tree + dirtiness.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir) + file_path.write_text("hello\ndirty\n") + result = run("git restore --staged f.py", payload_cwd=str(repo_dir)) + assert result.returncode == 0, f"stderr={result.stderr!r}" + assert result.stdout == "" + assert result.stderr == "" + + +def test_checkout_denied_when_payload_cwd_is_missing(tmp_path: Path) -> None: + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("git checkout -- f.py", payload_cwd=None) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert ".cwd" in payload["systemMessage"] + + +def test_checkout_denied_when_payload_cwd_is_not_a_git_repo(tmp_path: Path) -> None: + not_a_repo = tmp_path / "not-a-repo" + not_a_repo.mkdir() + result = run("git checkout -- f.py", payload_cwd=str(not_a_repo)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "not inside a git working tree" in payload["systemMessage"] + + +def test_checkout_allowed_on_unborn_head_with_no_conflicting_content(tmp_path: Path) -> None: + """A fresh repo with no commits yet has no HEAD to diff against -- + must fall back to the empty-tree hash rather than spuriously denying a + genuinely clean fresh repo (issue #1375's own Acceptance Criteria + Map).""" + repo_dir = tmp_path / "repo" + repo_dir.mkdir() + _git(repo_dir, "init", "-q") + result = run("git checkout -- x.txt", payload_cwd=str(repo_dir)) + assert result.returncode == 0, f"stderr={result.stderr!r}" + + +def test_checkout_denied_on_unborn_head_when_staged(tmp_path: Path) -> None: + """The empty-tree-hash fallback still denies when the target genuinely + differs from an empty tree (staged on a fresh, commit-less repo).""" + repo_dir = tmp_path / "repo" + repo_dir.mkdir() + _git(repo_dir, "init", "-q") + (repo_dir / "x.txt").write_text("hello\n") + _git(repo_dir, "add", "x.txt") + result = run("git checkout -- x.txt", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_checkout_with_tree_relocation_flag_denied_end_to_end(tmp_path: Path) -> None: + """The classifier's own denial (found before this wrapper step ever + runs a live git call) reaches the operator through the full shell + pipeline too: a `-C` global flag makes the wrapper's own fixed `.cwd` + unsound for this invocation.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run(f"git -C {repo_dir} checkout -- f.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + +def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: + """A real merge conflict (issue #1375's own Acceptance Criteria Map): + the deny message names a remedy that actually works mid-conflict + (`git checkout -m -- PATH`), not only `git stash` (which fails with + "needs merge" while a conflict is unresolved).""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="f.txt", content="line1\n") + _git(repo_dir, "checkout", "-q", "-b", "branch-a") + file_path.write_text("line1-a\n") + _git(repo_dir, "commit", "-q", "-am", "change on a") + _git(repo_dir, "checkout", "-q", "-b", "branch-main", "main") + file_path.write_text("line1-main\n") + _git(repo_dir, "commit", "-q", "-am", "change on main") + subprocess.run( + ["git", "merge", "branch-a", "-q"], + cwd=str(repo_dir), + capture_output=True, + text=True, + timeout=10, + env={**os.environ, "GIT_CONFIG_GLOBAL": os.devnull, "GIT_CONFIG_NOSYSTEM": "1"}, + ) + result = run("git checkout -- f.txt", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "git checkout -m" in payload["systemMessage"] From a5f221b7451099019af676829ae380979bf03bb1 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 10:05:59 +0000 Subject: [PATCH 03/46] test(hooks): close remaining patch-coverage gaps on checkout/restore paths codecov flagged 7 uncovered lines on PR #1380 (97.31% patch coverage, target 99.55%): the --no-worktree branch, the bare and fused forms of recurse-submodules, the plain boolean-flag vocabulary in git restore's flag walk, and the checkout/restore subcommand scan's own normal loop exit when only global flags precede the end of a segment with no subcommand following. Adds a targeted test for each, bringing hooks/gitapex_check_bash_safety.py to 100% line and branch coverage. Refs #1375. --- ...st_gitapex_check_bash_safety_properties.py | 56 +++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 4516aa27..232502fe 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2813,6 +2813,62 @@ def test_git_restore_paths_last_occurrence_wins_for_staged(last: str) -> None: assert resolved == ("f.py",) +@_PROPERTIES +@given(paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) +def test_git_restore_paths_last_occurrence_wins_for_worktree(paths: list[str]) -> None: + """Model-based: `saw_worktree` is last-occurrence-wins too -- + `--staged --worktree --no-worktree` ends with `saw_worktree=False`, + so the invocation is safe (staged, not worktree) and never + live-checked -- exercises the `--no-worktree` branch directly.""" + reason, resolved = checker._git_restore_paths(["--staged", "--worktree", "--no-worktree", *paths], {}) + assert reason is None + assert resolved == () + + +@_PROPERTIES +@given( + flag=st.sampled_from(sorted(checker._RESTORE_BOOLEAN_FLAGS)), + paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3), +) +def test_git_restore_paths_every_boolean_flag_consumes_no_value(flag: str, paths: list[str]) -> None: + """Model-based: every flag in the enumerated boolean vocabulary + (`--quiet`/`-q`, `--progress`/`--no-progress`, `--overlay`/ + `--no-overlay`, `--ours`/`--theirs`, `--merge`/`-m`, + `--ignore-unmerged`, `--ignore-skip-worktree-bits`) is skipped without + consuming the token after it as a value -- the following path tokens + are still extracted.""" + reason, resolved = checker._git_restore_paths([flag, *paths], {}) + assert reason is None + assert resolved == tuple(paths) + + +@_PROPERTIES +@given(value=st.sampled_from(["yes", "no"]), paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) +def test_git_restore_paths_recurse_submodules_bare_and_fused(value: str, paths: list[str]) -> None: + """Model-based: `--recurse-submodules` (bare, consumes nothing) and + `--recurse-submodules=VALUE` (fused, self-contained) are both skipped + without treating the next token as a value or as part of the flag.""" + reason, resolved = checker._git_restore_paths(["--recurse-submodules", *paths], {}) + assert reason is None + assert resolved == tuple(paths) + reason, resolved = checker._git_restore_paths([f"--recurse-submodules={value}", *paths], {}) + assert reason is None + assert resolved == tuple(paths) + + +def test_find_git_checkout_restore_none_when_only_global_flags_and_no_subcommand_follow() -> None: + """No false positive, and direct coverage for the flag-skip loop's own + normal (non-`break`, non-ambiguous) exit: `git -C /tmp/x` with global + flags consuming every remaining token and nothing left over is not a + checkout/restore invocation -- the while loop runs off the end of the + segment (`j == n`) rather than finding a literal `checkout`/`restore` + token.""" + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(["git", "-C", "/tmp/x"]) + assert subcommand is None + assert tokens_after == [] + assert saw_tree_relocation is False + + @_PROPERTIES @given(flag=st.sampled_from(["--pathspec-from-file=list.txt", "--pathspec-from-file", "--pathspec-file-nul"])) def test_git_restore_paths_denies_pathspec_from_file(flag: str) -> None: From 65f48365eb0f13423d7726ffb5f994eb0e2f8015 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 10:17:14 +0000 Subject: [PATCH 04/46] fix(hooks): close a vanishing-decoy bypass and a newline false-positive Independent adversarial review of PR #1380 found two real defects in the new git checkout/restore detection surface, both live-confirmed before being fixed here. Critical: a genuinely-unset, unquoted decoy token between "git" and "checkout"/"restore" (e.g. git $NEVERSET checkout -- file.py) made _find_git_checkout_restore treat that git occurrence as ambiguous and give up, so checkout_restore_paths came back empty and the command was silently allowed -- a near-zero-effort bypass of the entire feature, reproduced live via a real bash proxy confirming the decoy word-splits away to nothing and the command genuinely runs as plain "git checkout -- file.py". Fixed by reusing _token_is_all_unassigned_refs, the same primitive the existing git-push detector already relies on for the identical position, to skip a token that unambiguously vanishes at real bash runtime instead of giving up on it. Medium: shlex's own default whitespace set includes a bare newline, so tokenize() never produced a distinct newline token, making segment_tokens' own newline-boundary handling dead code. Every prior rule was unaffected since none of them depend on a segment actually ending where a line break falls, but the new checkout/restore path extraction consumes every token to the end of its segment, so an ordinary two-line script with a checkout on one line and something unrelated with a dynamic token on the next line had that second line's own token swept in and spuriously denied. Fixed by moving the newline into shlex's own punctuation_chars instead, so it tokenizes as its own operator token; a newline inside a quoted string still stays fused into its own token as before. Both are pinned with regression tests, and hooks/gitapex_check_bash_safety.py stays at 100% line and branch coverage. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 69 ++++++++++-- ...st_gitapex_check_bash_safety_properties.py | 101 ++++++++++++++++-- 2 files changed, 153 insertions(+), 17 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index a9b72cfc..9270acef 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -880,10 +880,36 @@ def tokenize(command: str) -> list[str]: spans here -- `_classify_tokens` applies `_fold_command_substitution_ spans` itself, AFTER first running `_rule_command_substitution_ content` against these still-unfolded tokens, which needs each - span's own inner tokens still separable.""" + span's own inner tokens still separable. + + An UNQUOTED newline is one of `segment_tokens`'s own documented + segment boundaries (it is a member of `_SINGLE_OPS`) -- but shlex's + own default `whitespace` set includes `\\n`, so a bare newline was + silently swallowed as ordinary inter-token whitespace and never + reached `segment_tokens` as its own token at all, making that half of + `_SINGLE_OPS` dead code. Found live by independent adversarial review + of issue #1375's own new checkout/restore detection: unlike every + prior rule in this module (none of which depend on a segment actually + ending where a real multi-line script's own line breaks fall), + `_git_checkout_paths`/`_git_restore_paths` consume every token up to + the (wrongly unbroken) end of the segment as candidate path data -- + confirmed live that `git checkout -b newbranch master\\necho + "exit=$?"` (an ordinary two-line script, checkout on the first line, + something unrelated on the second) had the second line's own + `exit=$?` swept in as a checkout path candidate and spuriously + denied the whole command as an unresolvable dynamic path, purely + because the newline between the two lines was never recognized as a + boundary. Closed by moving `\\n` from shlex's own `whitespace` set + into `punctuation_chars` instead, so it tokenizes as its own + single-character operator token -- confirmed live this does not + change a QUOTED newline (still preserved verbatim inside its + enclosing token, since shlex's own quote handling runs before + punctuation-splitting) nor any command with no literal newline in it + at all (the entire pre-existing test suite's own command strings).""" try: - lexer = shlex.shlex(command, posix=True, punctuation_chars=True) + lexer = shlex.shlex(command, posix=True, punctuation_chars="();<>|&\n") lexer.whitespace_split = True + lexer.whitespace = lexer.whitespace.replace("\n", "") raw_tokens = list(lexer) except ValueError as error: raise TokenizeError(str(error)) from error @@ -2689,7 +2715,7 @@ def _git_restore_paths( return _resolve_path_tokens(path_tokens, name_to_raw_value) -def _find_git_checkout_restore(seg: list[str]) -> tuple[str | None, list[str], bool]: +def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str]) -> tuple[str | None, list[str], bool]: """Scan SEG (already assignment-stripped, see `_strip_leading_ assignments`) for a `git checkout`/`git restore` invocation, skipping past git's own global value-taking options the same way @@ -2711,17 +2737,35 @@ def _find_git_checkout_restore(seg: list[str]) -> tuple[str | None, list[str], b Acceptance Criteria Map names this exact shape) through with an empty `checkout_restore_paths` instead of denying on the unresolvable `$f`. + A DYNAMIC token encountered while skip-parsing global flags is + resolved via `_token_is_all_unassigned_refs` (the same primitive + `_is_git_push_segment` already uses for the identical position) rather + than always treated as ambiguous: a token that unambiguously vanishes + at real bash runtime (a genuinely unset, unquoted `$NEVERSET`-shaped + reference) is skipped over, not given up on. Found live by + independent adversarial review of this PR: `git $NEVERSET checkout -- + file.py` (NEVERSET never assigned) was wrongly allowed with an empty + `checkout_restore_paths` before this check -- confirmed live via a + real bash proxy (stand-in `git` binary on PATH, capturing its own + argv) that the decoy word-splits away to nothing and this genuinely + runs `git checkout -- file.py`, silently bypassing the whole feature + with one trivial unset variable. A dynamic token that does NOT + unambiguously vanish (assigned, or an indirect/default-clause + reference this primitive does not cover) still makes this `git` + occurrence ambiguous, per the paragraph below. + Returns `(subcommand, tokens_after_subcommand, saw_tree_relocation_flag)`; `subcommand` is `None` when SEG has no checkout/restore invocation at all (including when a dynamic token sits in a position that could be either a global flag/value or the - subcommand itself, immediately after a literal `git` -- a genuinely - ambiguous, non-honest-accident shape this pure classifier declines to - resolve at that specific `git` occurrence, the same disclosed-residual - convention this module's own `KNOWN_BYPASS_COMMANDS` test list already - uses for the analogous dynamic-tool/dynamic-verb case; scanning - continues past it to any later `git` occurrence in the same segment), - in which case the other two return values are meaningless.""" + subcommand itself, immediately after a literal `git`, and does not + unambiguously vanish -- a genuinely ambiguous, non-honest-accident + shape this pure classifier declines to resolve at that specific `git` + occurrence, the same disclosed-residual convention this module's own + `KNOWN_BYPASS_COMMANDS` test list already uses for the analogous + dynamic-tool/dynamic-verb case; scanning continues past it to any + later `git` occurrence in the same segment), in which case the other + two return values are meaningless.""" n = len(seg) for i, tok in enumerate(seg): if _is_dynamic(tok) or tok.lower() != "git": @@ -2732,6 +2776,9 @@ def _find_git_checkout_restore(seg: list[str]) -> tuple[str | None, list[str], b while j < n: candidate = seg[j] if _is_dynamic(candidate): + if _token_is_all_unassigned_refs(candidate, name_to_raw_value): + j += 1 + continue ambiguous = True break if ( @@ -2774,7 +2821,7 @@ def _rule_git_checkout_restore( saw_cd = False all_paths: list[str] = [] for seg in segments: - subcommand, tokens_after, saw_tree_relocation = _find_git_checkout_restore(seg) + subcommand, tokens_after, saw_tree_relocation = _find_git_checkout_restore(seg, raw_assigned) if subcommand is None: if any(not _is_dynamic(t) and t == "cd" for t in seg): saw_cd = True diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 232502fe..fe82dfdc 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -20,6 +20,7 @@ from __future__ import annotations import json +import shlex import string import sys from typing import cast @@ -2863,7 +2864,7 @@ def test_find_git_checkout_restore_none_when_only_global_flags_and_no_subcommand checkout/restore invocation -- the while loop runs off the end of the segment (`j == n`) rather than finding a literal `checkout`/`restore` token.""" - subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(["git", "-C", "/tmp/x"]) + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(["git", "-C", "/tmp/x"], {}) assert subcommand is None assert tokens_after == [] assert saw_tree_relocation is False @@ -2909,7 +2910,7 @@ def test_find_git_checkout_restore_finds_git_at_any_segment_position(prefix: lis found at `seg[0]`.""" assume(all(p != "git" for p in prefix)) seg = [*prefix, "git", "checkout", "--", *paths] - subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg) + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}) assert subcommand == "checkout" assert tokens_after == ["--", *paths] assert saw_tree_relocation is False @@ -2918,7 +2919,7 @@ def test_find_git_checkout_restore_finds_git_at_any_segment_position(prefix: lis def test_find_git_checkout_restore_none_for_a_segment_with_no_git() -> None: """No false positive: a segment with no literal `git` token at all is never treated as a checkout/restore invocation.""" - subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["echo", "checkout", "restore"]) + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["echo", "checkout", "restore"], {}) assert subcommand is None @@ -2931,7 +2932,7 @@ def test_find_git_checkout_restore_flags_tree_relocation(flag: str) -> None: rather than let the live wrapper check the wrong tree (issue #1375's own Fact 5 cwd finding).""" seg = ["git", flag, "/some/path", "checkout", "--", "f.py"] - subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg) + subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}) assert subcommand == "checkout" assert saw_tree_relocation is True @@ -2941,7 +2942,7 @@ def test_find_git_checkout_restore_does_not_flag_lowercase_c_config_flag() -> No case-sensitively distinct from `-C` (uppercase, relocates the working tree) and must never be conflated with it (issue #1375's own Fact 5).""" seg = ["git", "-c", "user.name=x", "checkout", "--", "f.py"] - subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg) + subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}) assert subcommand == "checkout" assert saw_tree_relocation is False @@ -2952,10 +2953,98 @@ def test_find_git_checkout_restore_is_a_non_goal_for_a_dynamic_subcommand() -> N shaped and is not detected -- the same disclosed-residual convention this module's own `KNOWN_BYPASS_COMMANDS` test list already uses for the analogous dynamic-tool/dynamic-verb case.""" - subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["git", "$V", "--", "f.py"]) + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["git", "$V", "--", "f.py"], {}) assert subcommand is None +# --- Regression pins for two real findings from this PR's own independent +# adversarial review (issue #1375), both confirmed live before being fixed. + + +def test_find_git_checkout_restore_skips_a_vanishing_decoy_between_git_and_subcommand() -> None: + """CRITICAL regression pin. A genuinely-unset, unquoted `$NEVERSET` + sitting between `git` and `checkout`/`restore` word-splits away to + nothing at real bash runtime (confirmed live via a real bash proxy, + stand-in `git` binary on PATH, capturing its own argv: `git $NEVERSET + checkout -- file.py` genuinely runs `git checkout -- file.py`). Before + this fix, `_find_git_checkout_restore` treated ANY dynamic token in + this position as ambiguous and gave up, so this exact near-zero-effort + decoy silently bypassed the entire checkout/restore safety feature -- + the same vanishing-decoy bug class `_is_git_push_segment` already + closed for `git push` over rounds 20-24 of issue #1326, using the same + `_token_is_all_unassigned_refs` primitive this fix now reuses here.""" + seg = ["git", "$NEVERSET", "checkout", "--", "file.py"] + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}) + assert subcommand == "checkout" + assert tokens_after == ["--", "file.py"] + assert saw_tree_relocation is False + + +def test_classify_denies_checkout_with_a_vanishing_decoy_when_dirty() -> None: + """End-to-end regression pin for the same finding: `classify()` must + surface the resolved path, not silently allow with an empty + `checkout_restore_paths`, when a vanishing decoy sits between `git` + and `checkout`.""" + verdict = checker.classify("git $NEVERSET checkout -- file.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("file.py",) + + +def test_find_git_checkout_restore_does_not_skip_an_assigned_dynamic_token() -> None: + """No false positive from the vanishing-decoy fix itself: a dynamic + token that IS assigned a real (non-empty) value does not + unambiguously vanish, so it still makes this `git` occurrence + ambiguous -- unchanged from the pre-fix behavior for this case.""" + seg = ["git", "$SET", "checkout", "--", "file.py"] + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(seg, {"SET": "-C"}) + assert subcommand is None + + +def test_tokenize_splits_on_an_unquoted_newline() -> None: + """MEDIUM regression pin. shlex's own default `whitespace` set + includes `\\n`, silently swallowing an unquoted newline as ordinary + inter-token whitespace and never producing it as its own token -- + making `_SINGLE_OPS`'s inclusion of `"\\n"` (and `segment_tokens`'s + own documented newline-boundary behavior) dead code. Confirmed live + that this let an ordinary two-line script (`git checkout` on one + line, something unrelated with a `$` token on the next) get the + second line's own token swept into the first line's own checkout + path candidates and spuriously denied -- not a security miss (the + fail-closed direction), but a real false-positive regression for a + very common multi-line Bash tool-call shape.""" + tokens = checker.tokenize("echo a\necho b") + assert tokens == ["echo", "a", "\n", "echo", "b"] + + +def test_tokenize_preserves_a_newline_inside_a_quoted_string() -> None: + """No false positive from the newline fix itself: a newline INSIDE a + quoted string is real string content, not a shell control operator, + and must stay fused into its own token exactly as before.""" + tokens = checker.tokenize('echo "multi\nline"') + assert tokens == ["echo", "multi\nline"] + + +def test_classify_does_not_leak_a_later_lines_token_into_an_earlier_checkout() -> None: + """End-to-end regression pin for the newline finding: an ordinary + two-line script with `git checkout` on the first line and an + unrelated `$`-containing token on the second line must classify the + checkout using only its own line's tokens.""" + verdict = checker.classify('git checkout -b newbranch master\necho "exit=$?"') + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("newbranch", "master") + + +def test_shlex_default_punctuation_chars_still_matches_the_hardcoded_extension() -> None: + """Pins the stdlib assumption `tokenize()`'s own newline fix depends + on: shlex's documented default `punctuation_chars` value for + `punctuation_chars=True`, which `tokenize()` now hardcodes (extended + with `\\n`) rather than deriving at runtime. If a future Python + version ever changes this default, this test fails loudly instead of + `tokenize()` silently drifting from it.""" + default_lexer = shlex.shlex("x", posix=True, punctuation_chars=True) + assert default_lexer.punctuation_chars == "();<>|&" + + @_PROPERTIES @given(command_paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_paths: list[str]) -> None: From ffb6321019156a85cd8c875e0db296add58e1d53 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 10:32:15 +0000 Subject: [PATCH 05/46] fix(hooks): close an empty-default-clause bypass and two restore false positives A second, independent round of adversarial review of PR #1380 found the round-1 vanishing-decoy fix does not cover every shape that vanishes. _token_is_all_unassigned_refs deliberately excludes any default-clause reference on the reasoning that a default clause always supplies real substitute text -- true when that text is non-empty, but not when it is literally empty. ${NEVERSET:-}, ${NEVERSET-}, ${NEVERSET:+x}, and ${NEVERSET+x} (NEVERSET never assigned) all confirmed live via a real bash proxy to word-split away to nothing exactly like a bare $NEVERSET, reopening the same near-zero-effort bypass the first fix closed, just spelled differently. Rather than widen the shared, already many-times-revised _token_is_all_unassigned_refs primitive (used by several existing rules, with a long history of narrow fixes and reverted over-generalizations), this adds a small, local helper scoped to the checkout/restore detector that recognizes exactly these two clause shapes, delegating the real "does NAME itself vanish" question back to the existing primitive on a synthesized plain reference rather than re-deriving it. The same review round also found two low-severity false positives in git restore's own flag walk: a literal -- (valid git syntax disambiguating a pathspec, the same role it already plays for git checkout) and the fused --source=VALUE/--conflict=VALUE forms were both denied outright as unrecognized flags. Both are now recognized. All three are pinned with regression tests, including a property test exercising the new helper directly per the repository's own detection- logic property-coverage gate, and hooks/gitapex_check_bash_safety.py stays at 100% line and branch coverage. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 97 ++++++++++++++- ...st_gitapex_check_bash_safety_properties.py | 117 ++++++++++++++++++ 2 files changed, 213 insertions(+), 1 deletion(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 9270acef..aa515cbf 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -2670,6 +2670,17 @@ def _git_restore_paths( n = len(tokens_after) while i < n: tok = tokens_after[i] + if tok == "--": + # Real git syntax: `--` disambiguates every remaining token as + # a pathspec, the identical role it plays for `git checkout` + # (see `_git_checkout_paths`'s own sub-case (a)). Found by + # independent adversarial review of this PR: the pre-fix + # version had no case for a literal `--` at all, so it fell + # into the unrecognized-flag branch below and denied an + # entirely ordinary, harmless `git restore -- PATH` outright. + path_tokens.extend(tokens_after[i + 1 :]) + i = n + break if tok == "--pathspec-from-file" or tok.startswith("--pathspec-from-file=") or tok == "--pathspec-file-nul": return ( "a 'git restore --pathspec-from-file'/'--pathspec-file-nul' flag reads paths from a file this " @@ -2701,6 +2712,15 @@ def _git_restore_paths( if tok in _RESTORE_VALUE_FLAGS: i += 2 continue + if any(tok.startswith(f"{flag}=") for flag in _RESTORE_VALUE_FLAGS): + # Fused `--source=main`/`--conflict=diff3`: self-contained, + # unlike the separate-token form above -- no extra token to + # skip. Found by independent adversarial review of this PR: + # the pre-fix version only recognized the bare, separate-token + # spelling and denied this equally legitimate fused one as an + # unrecognized flag. + i += 1 + continue if tok.startswith("-"): return ( f"an unrecognized 'git restore' flag ({tok!r}) -- this classifier cannot safely guarantee " @@ -2715,6 +2735,79 @@ def _git_restore_paths( return _resolve_path_tokens(path_tokens, name_to_raw_value) +# A whole token that is EXACTLY one `${NAME-}`/`${NAME:-}` (empty default +# text) or `${NAME+anything}`/`${NAME:+anything}` (alternate-value clause) +# construct -- nothing else fused in. Deliberately narrower than +# `_ONE_REF_SRC`'s own general reference-run matching (this is a single, +# whole-token check, not a "run of references" one): the two clause shapes +# below need their own NAME extracted and their own vanishing rule applied +# (see `_token_is_a_vanishing_default_or_alt_clause`'s own docstring), +# unlike a bare/braced/subscript reference where any run of them can be +# fused together and each one either independently vanishes or doesn't. +_EMPTY_DEFAULT_CLAUSE_RE = re.compile(r"^\$\{(?P[A-Za-z_][A-Za-z0-9_]*):?-\}$") +_ALT_VALUE_CLAUSE_RE = re.compile(r"^\$\{(?P[A-Za-z_][A-Za-z0-9_]*)(?P:)?\+[^}]*\}$") + + +def _token_is_a_vanishing_default_or_alt_clause(token: str, name_to_raw_value: dict[str, str]) -> bool: + """TOKEN word-splits away to NOTHING, unquoted, at real bash runtime, + because it is an `${NAME-}`/`${NAME:-}` (empty default) or + `${NAME+word}`/`${NAME:+word}` (alternate-value) construct whose own + substitution is empty. Deliberately narrow and LOCAL to this module's + own checkout/restore detection (issue #1375) rather than folded into + `_token_is_all_unassigned_refs` itself: that function's own docstring + explicitly and, for a NON-empty default/alt text, CORRECTLY excludes + every default-clause shape ("a default-clause reference supplies REAL + substitute text regardless of whether NAME is assigned, so it never + vanishes to nothing") -- widening that already heavily-scrutinized, + many-times-revised shared primitive (28+ documented rounds of narrow + fixes and reverted over-generalizations, per its own docstring) is a + materially larger, riskier change than this specific, live-confirmed + gap warrants; every existing caller of that function keeps its exact + prior behavior unchanged. + + Found live by independent adversarial review of this PR, in the SAME + position the already-fixed `$NEVERSET`-shaped bug occupied: that + function's own blanket exclusion is only sound when the default/alt + text is non-empty -- `${NEVERSET:-}`, `${NEVERSET-}`, and + `${NEVERSET:+x}` (NEVERSET genuinely never assigned) all confirmed + live (a real bash proxy capturing argv) to word-split away to nothing + identically to a bare `$NEVERSET`, making `git ${NEVERSET:-} checkout + -- file.py` genuinely run as `git checkout -- file.py` -- the same + near-zero-effort bypass of the entire feature the bare-reference fix + closed, in a shape that fix's own primitive does not recognize. + + Two sound, narrow cases, both delegating the actual "does NAME itself + vanish" question back to `_token_is_all_unassigned_refs` on a + synthesized plain `${NAME}` reference (reusing its own already-correct, + already-tested per-name rule -- assigned-empty and assigned-all-IFS- + whitespace both count as vanishing there too -- rather than + re-deriving it here): + - `${NAME-}`/`${NAME:-}` (default text is the empty string, checked + via the regex itself, not a resolved value): the whole construct + substitutes NAME's own value if NAME is set (colon form: set AND + non-empty), else the empty default text -- either way, this + construct vanishes exactly when NAME itself does. + - `${NAME:+word}` (colon form): substitutes WORD only when NAME is set + AND non-empty, else nothing -- so this construct vanishes exactly + when NAME itself does, regardless of WORD's own content (WORD is + never evaluated in the vanishing branch). + - `${NAME+word}` (no-colon form): substitutes WORD when NAME is set at + ALL (even assigned-empty), else nothing -- a stricter condition than + `_token_is_all_unassigned_refs`'s own "set but empty/IFS-whitespace + still counts as vanishing," so this form is only recognized when + NAME is not a key in NAME_TO_RAW_VALUE at all, not delegated to that + broader check.""" + match = _EMPTY_DEFAULT_CLAUSE_RE.match(token) + if match: + return _token_is_all_unassigned_refs(f"${{{match.group('name')}}}", name_to_raw_value) + match = _ALT_VALUE_CLAUSE_RE.match(token) + if match: + if match.group("colon"): + return _token_is_all_unassigned_refs(f"${{{match.group('name')}}}", name_to_raw_value) + return match.group("name") not in name_to_raw_value + return False + + def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str]) -> tuple[str | None, list[str], bool]: """Scan SEG (already assignment-stripped, see `_strip_leading_ assignments`) for a `git checkout`/`git restore` invocation, skipping @@ -2776,7 +2869,9 @@ def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str] while j < n: candidate = seg[j] if _is_dynamic(candidate): - if _token_is_all_unassigned_refs(candidate, name_to_raw_value): + if _token_is_all_unassigned_refs( + candidate, name_to_raw_value + ) or _token_is_a_vanishing_default_or_alt_clause(candidate, name_to_raw_value): j += 1 continue ambiguous = True diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index fe82dfdc..ae878e03 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2870,6 +2870,45 @@ def test_find_git_checkout_restore_none_when_only_global_flags_and_no_subcommand assert saw_tree_relocation is False +# --- Two LOW-severity false positives, found by the same independent +# adversarial review round that found the round-2 vanishing-decoy bypass +# above: `git restore` denied two entirely legitimate, harmless +# invocations outright as "unrecognized flag" because neither shape was in +# the enumerated vocabulary. + + +@_PROPERTIES +@given(paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) +def test_git_restore_paths_extracts_every_token_after_double_dash(paths: list[str]) -> None: + """`--` disambiguates every remaining token as a pathspec for `git + restore`, the identical role it plays for `git checkout` -- must be + recognized, not denied as an unrecognized flag.""" + reason, resolved = checker._git_restore_paths(["--", *paths], {}) + assert reason is None + assert resolved == tuple(paths) + + +@_PROPERTIES +@given( + flag_value=st.sampled_from([("--source", "main"), ("--conflict", "diff3")]), + paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3), +) +def test_git_restore_paths_recognizes_fused_value_flags(flag_value: tuple[str, str], paths: list[str]) -> None: + """`--source=VALUE`/`--conflict=VALUE` (fused with `=`) are equally + legitimate git syntax as the separate-token form already recognized -- + must not be denied as an unrecognized flag.""" + flag, value = flag_value + reason, resolved = checker._git_restore_paths([f"{flag}={value}", *paths], {}) + assert reason is None + assert resolved == tuple(paths) + + +def test_classify_allows_restore_double_dash_when_clean() -> None: + verdict = checker.classify("git restore -- f.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("f.py",) + + @_PROPERTIES @given(flag=st.sampled_from(["--pathspec-from-file=list.txt", "--pathspec-from-file", "--pathspec-file-nul"])) def test_git_restore_paths_denies_pathspec_from_file(flag: str) -> None: @@ -2990,6 +3029,84 @@ def test_classify_denies_checkout_with_a_vanishing_decoy_when_dirty() -> None: assert verdict.checkout_restore_paths == ("file.py",) +# --- Round 2: an empty-default/alt-clause decoy in the SAME position, +# found by a second, independent adversarial review pass after the first +# fix above landed. `${NEVERSET:-}`/`${NEVERSET-}`/`${NEVERSET:+x}` all +# vanish identically to a bare `$NEVERSET` when NEVERSET is genuinely +# unset (confirmed live via a real bash proxy), but `_token_is_all_ +# unassigned_refs`'s own regex never matches these clause shapes at all -- +# its own docstring deliberately excludes them for a non-empty default, +# which is correct, but does not carve out the empty-default case. + + +@_PROPERTIES +@given(clause=st.sampled_from(["${NEVERSET:-}", "${NEVERSET-}", "${NEVERSET:+x}", "${NEVERSET+x}"])) +def test_classify_denies_checkout_with_an_empty_default_or_alt_clause_decoy(clause: str) -> None: + """CRITICAL regression pin, round 2. Every one of these four clause + shapes, with NEVERSET genuinely never assigned, must be recognized as + vanishing -- the same near-zero-effort bypass class as the bare + `$NEVERSET` case, just spelled differently.""" + verdict = checker.classify(f"git {clause} checkout -- file.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("file.py",) + + +@_PROPERTIES +@given(name=_IDENTIFIERS, shape=st.sampled_from(["${{{name}:-}}", "${{{name}-}}", "${{{name}:+x}}", "${{{name}+x}}"])) +def test_token_is_a_vanishing_default_or_alt_clause_true_for_any_unassigned_name(name: str, shape: str) -> None: + """Model-based, direct coverage of `_token_is_a_vanishing_default_or_ + alt_clause` itself: for ANY identifier never assigned, all four + empty-default/alt-clause shapes are recognized as vanishing.""" + token = shape.format(name=name) + assert checker._token_is_a_vanishing_default_or_alt_clause(token, {}) is True + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_VALUES) +def test_token_is_a_vanishing_default_or_alt_clause_false_for_any_assigned_non_empty_name( + name: str, value: str +) -> None: + """Model-based: for ANY identifier assigned a real, non-empty value, + the colon-form clauses never vanish.""" + assert checker._token_is_a_vanishing_default_or_alt_clause(f"${{{name}:-}}", {name: value}) is False + assert checker._token_is_a_vanishing_default_or_alt_clause(f"${{{name}:+x}}", {name: value}) is False + + +def test_token_is_a_vanishing_default_or_alt_clause_true_for_empty_default_unassigned() -> None: + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET:-}", {}) is True + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET-}", {}) is True + + +def test_token_is_a_vanishing_default_or_alt_clause_true_for_alt_clause_unassigned() -> None: + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET:+x}", {}) is True + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET+x}", {}) is True + + +def test_token_is_a_vanishing_default_or_alt_clause_false_for_non_empty_default() -> None: + """No false positive: a NON-empty default text supplies real + substitute text regardless of NAME's own state, so it never + vanishes.""" + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET:-x}", {}) is False + + +def test_token_is_a_vanishing_default_or_alt_clause_false_when_name_is_assigned_non_empty() -> None: + """No false positive: a NAME assigned a real, non-empty value does not + vanish under the colon forms.""" + assert checker._token_is_a_vanishing_default_or_alt_clause("${SET:-}", {"SET": "-C"}) is False + assert checker._token_is_a_vanishing_default_or_alt_clause("${SET:+x}", {"SET": "-C"}) is False + + +def test_token_is_a_vanishing_default_or_alt_clause_no_colon_plus_requires_strictly_unset() -> None: + """The no-colon `+` form checks "is NAME set AT ALL" (ignoring + emptiness), a stricter condition than the colon form's "set and + non-empty" -- a NAME assigned the empty string still counts as SET + for this form, so `${NAME+word}` does NOT vanish (WORD is genuinely + substituted at real bash runtime), unlike `${NAME:+word}` for the + identical assigned-empty NAME.""" + assert checker._token_is_a_vanishing_default_or_alt_clause("${SET+x}", {"SET": ""}) is False + assert checker._token_is_a_vanishing_default_or_alt_clause("${SET:+x}", {"SET": ""}) is True + + def test_find_git_checkout_restore_does_not_skip_an_assigned_dynamic_token() -> None: """No false positive from the vanishing-decoy fix itself: a dynamic token that IS assigned a real (non-empty) value does not From f48a7fa4f7c9dea816aa65ae67108804c7782c56 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 10:35:18 +0000 Subject: [PATCH 06/46] fix(hooks): close the assign-default variant of the same vanishing decoy Self-review after the round-2 fix found one more shape of the identical bug: bash's assign-default parameter expansion, ${NAME:=} and ${NAME=} with empty text, also word-splits away to nothing when NAME is unassigned (confirmed live via a real bash proxy), the same as the already-fixed ${NAME:-}/${NAME-} default-clause forms, but the helper only matched the "-" operator, not "=". Widens the same local regex to accept both operators. Also adds an explicit no-false-positive pin for ${NAME:?}/${NAME?} (the error-message clause): unlike every other clause here, real bash terminates the whole command with an error when NAME is unset for that one, so it is correctly left unrecognized rather than treated as vanishing -- there is no real invocation for a missed detection to miss. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 40 ++++++++++------ ...st_gitapex_check_bash_safety_properties.py | 46 +++++++++++++++---- 2 files changed, 65 insertions(+), 21 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index aa515cbf..5d56b875 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -2736,24 +2736,38 @@ def _git_restore_paths( # A whole token that is EXACTLY one `${NAME-}`/`${NAME:-}` (empty default -# text) or `${NAME+anything}`/`${NAME:+anything}` (alternate-value clause) -# construct -- nothing else fused in. Deliberately narrower than -# `_ONE_REF_SRC`'s own general reference-run matching (this is a single, -# whole-token check, not a "run of references" one): the two clause shapes -# below need their own NAME extracted and their own vanishing rule applied -# (see `_token_is_a_vanishing_default_or_alt_clause`'s own docstring), -# unlike a bare/braced/subscript reference where any run of them can be -# fused together and each one either independently vanishes or doesn't. -_EMPTY_DEFAULT_CLAUSE_RE = re.compile(r"^\$\{(?P[A-Za-z_][A-Za-z0-9_]*):?-\}$") +# text), `${NAME=}`/`${NAME:=}` (empty ASSIGN-default text -- also +# assigns NAME the empty string as a side effect, which does not change +# whether THIS token itself vanishes), or `${NAME+anything}`/ +# `${NAME:+anything}` (alternate-value clause) construct -- nothing else +# fused in. Deliberately narrower than `_ONE_REF_SRC`'s own general +# reference-run matching (this is a single, whole-token check, not a "run +# of references" one): the clause shapes below need their own NAME +# extracted and their own vanishing rule applied (see +# `_token_is_a_vanishing_default_or_alt_clause`'s own docstring), unlike a +# bare/braced/subscript reference where any run of them can be fused +# together and each one either independently vanishes or doesn't. +# +# `${NAME:?}`/`${NAME?}` (empty error-message clause) is deliberately NOT +# included here: unlike every clause above, this one does not silently +# vanish when NAME is unset -- real bash prints the message to stderr and +# TERMINATES the command entirely (non-interactively) with a non-zero +# status, confirmed live, so `checkout`/`restore` never even runs. Treating +# it as ambiguous (the default, unrecognized-dynamic-token fallback) is +# already safe: there is no real invocation for a missed detection to miss. +_EMPTY_DEFAULT_CLAUSE_RE = re.compile(r"^\$\{(?P[A-Za-z_][A-Za-z0-9_]*):?[-=]\}$") _ALT_VALUE_CLAUSE_RE = re.compile(r"^\$\{(?P[A-Za-z_][A-Za-z0-9_]*)(?P:)?\+[^}]*\}$") def _token_is_a_vanishing_default_or_alt_clause(token: str, name_to_raw_value: dict[str, str]) -> bool: """TOKEN word-splits away to NOTHING, unquoted, at real bash runtime, - because it is an `${NAME-}`/`${NAME:-}` (empty default) or - `${NAME+word}`/`${NAME:+word}` (alternate-value) construct whose own - substitution is empty. Deliberately narrow and LOCAL to this module's - own checkout/restore detection (issue #1375) rather than folded into + because it is an `${NAME-}`/`${NAME:-}` (empty default), `${NAME=}`/ + `${NAME:=}` (empty assign-default -- confirmed live this also + vanishes to nothing the identical way, the assignment side effect + notwithstanding), or `${NAME+word}`/`${NAME:+word}` (alternate-value) + construct whose own substitution is empty. Deliberately narrow and + LOCAL to this module's own checkout/restore detection (issue #1375) + rather than folded into `_token_is_all_unassigned_refs` itself: that function's own docstring explicitly and, for a NON-empty default/alt text, CORRECTLY excludes every default-clause shape ("a default-clause reference supplies REAL diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index ae878e03..160bc91e 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3040,23 +3040,34 @@ def test_classify_denies_checkout_with_a_vanishing_decoy_when_dirty() -> None: @_PROPERTIES -@given(clause=st.sampled_from(["${NEVERSET:-}", "${NEVERSET-}", "${NEVERSET:+x}", "${NEVERSET+x}"])) +@given( + clause=st.sampled_from( + ["${NEVERSET:-}", "${NEVERSET-}", "${NEVERSET:=}", "${NEVERSET=}", "${NEVERSET:+x}", "${NEVERSET+x}"] + ) +) def test_classify_denies_checkout_with_an_empty_default_or_alt_clause_decoy(clause: str) -> None: - """CRITICAL regression pin, round 2. Every one of these four clause - shapes, with NEVERSET genuinely never assigned, must be recognized as - vanishing -- the same near-zero-effort bypass class as the bare - `$NEVERSET` case, just spelled differently.""" + """CRITICAL regression pin, round 2 (plus the `${NAME:=}`/`${NAME=}` + assign-default shapes found immediately afterward, same root cause). + Every one of these six clause shapes, with NEVERSET genuinely never + assigned, must be recognized as vanishing -- the same near-zero-effort + bypass class as the bare `$NEVERSET` case, just spelled differently.""" verdict = checker.classify(f"git {clause} checkout -- file.py") assert verdict.deny is False assert verdict.checkout_restore_paths == ("file.py",) @_PROPERTIES -@given(name=_IDENTIFIERS, shape=st.sampled_from(["${{{name}:-}}", "${{{name}-}}", "${{{name}:+x}}", "${{{name}+x}}"])) +@given( + name=_IDENTIFIERS, + shape=st.sampled_from( + ["${{{name}:-}}", "${{{name}-}}", "${{{name}:=}}", "${{{name}=}}", "${{{name}:+x}}", "${{{name}+x}}"] + ), +) def test_token_is_a_vanishing_default_or_alt_clause_true_for_any_unassigned_name(name: str, shape: str) -> None: """Model-based, direct coverage of `_token_is_a_vanishing_default_or_ - alt_clause` itself: for ANY identifier never assigned, all four - empty-default/alt-clause shapes are recognized as vanishing.""" + alt_clause` itself: for ANY identifier never assigned, all six + empty-default/assign-default/alt-clause shapes are recognized as + vanishing.""" token = shape.format(name=name) assert checker._token_is_a_vanishing_default_or_alt_clause(token, {}) is True @@ -3077,6 +3088,25 @@ def test_token_is_a_vanishing_default_or_alt_clause_true_for_empty_default_unass assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET-}", {}) is True +def test_token_is_a_vanishing_default_or_alt_clause_true_for_empty_assign_default_unassigned() -> None: + """`${NAME:=}`/`${NAME=}` (assign-default, empty text) also vanishes + to nothing when NAME is unassigned -- confirmed live that this both + substitutes the empty string AND assigns NAME the empty string as a + side effect, but the side effect does not change whether THIS token + itself occupies an argv position.""" + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET:=}", {}) is True + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET=}", {}) is True + + +def test_token_is_a_vanishing_default_or_alt_clause_false_for_error_message_clause() -> None: + """No false positive: `${NAME:?}`/`${NAME?}` is deliberately NOT + recognized -- real bash terminates the whole command with an error + when NAME is unset for this clause, rather than silently vanishing, + so there is no real invocation for a missed detection to miss.""" + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET:?}", {}) is False + assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET?}", {}) is False + + def test_token_is_a_vanishing_default_or_alt_clause_true_for_alt_clause_unassigned() -> None: assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET:+x}", {}) is True assert checker._token_is_a_vanishing_default_or_alt_clause("${NEVERSET+x}", {}) is True From c9be5b359779ee1ff87eef99a4936db1d745df1b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 10:38:10 +0000 Subject: [PATCH 07/46] docs(hooks): pin the remaining exotic-parameter-expansion residual Three rounds of adversarial review progressively closed every ordinary, honest-accident-shaped bash idiom for a decoy token between "git" and "checkout"/"restore" that vanishes at real bash runtime: a bare reference, and the default/assign-default/alt-value clause forms. Bash's other parameter-expansion operators that also evaluate to the empty string on an unset variable -- substring, prefix/suffix removal, pattern substitution, case modification -- are not honest-accident-shaped the way those are, matching this file's own established convention for exotic non-literal indirection (issue #1375's own Non-goals section). Pins this explicitly as a disclosed, tested residual rather than leaving it silently uncovered, following the same KNOWN_BYPASS_COMMANDS convention this file already uses for two other disclosed gaps: the decoy is correctly treated as ambiguous, not silently misread as a real checkout/restore invocation, so this is the same disclosed-residual shape the dynamic-subcommand Non-goal already carries, not a distinct or worse failure mode. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 14 ++++++++++++ hooks/test_gitapex_check_bash_safety.py | 29 +++++++++++++++++++++++++ 2 files changed, 43 insertions(+) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 5d56b875..e1f3a7bc 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -2521,6 +2521,20 @@ def _is_git_push_segment(seg: list[str], name_to_raw_value: dict[str, str]) -> b # `git diff --quiet HEAD -- PATH` exits 0 (clean) for a path that does not # exist, so treating an unresolved token as "nothing to check" would be # fail-OPEN, not fail-closed (confirmed live, git 2.43.0). +# +# Disclosed residual, matching this module's own established convention +# (see the module docstring's own "Known, disclosed limitation" paragraph +# above): a decoy token between `git` and `checkout`/`restore` that +# vanishes via a bash parameter-expansion operator OTHER than a bare +# reference or the default/assign-default/alt-value clauses (`${NAME:-}`/ +# `${NAME-}`, `${NAME:=}`/`${NAME=}`, `${NAME:+x}`/`${NAME+x}`) -- e.g. +# substring expansion, prefix/suffix removal, pattern substitution, or +# case modification, all of which also evaluate to the empty string on an +# unset variable -- is not recognized as vanishing; that `git` occurrence +# is correctly treated as ambiguous rather than silently misread as a safe +# checkout/restore. Pinned as `checkout-restore-exotic-parameter- +# expansion-decoy` in hooks/test_gitapex_check_bash_safety.py's own +# `KNOWN_BYPASS_COMMANDS`. _GIT_TREE_RELOCATION_LONG_FLAGS = {"--git-dir", "--work-tree"} _GIT_GLOBAL_SHORT_VALUE_FLAGS = {"-c", "-C"} diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 172e5689..8c8452ba 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -365,6 +365,35 @@ def assert_allowed(command: str) -> None: 'A=muta; B=tion; Q="${A}${B} { x }"; gh api graphql -f query="$Q"', "graphql-mutation-keyword-variable-concatenation", ), + ( + # Issue #1375's own checkout/restore path extraction closes every + # ORDINARY, honest-accident-shaped way a decoy token between `git` + # and `checkout`/`restore` vanishes at real bash runtime: a bare + # `$NAME`/`${NAME}` reference, and the default/assign-default/ + # alt-value clause forms (`${NAME:-}`/`${NAME-}`, `${NAME:=}`/ + # `${NAME=}`, `${NAME:+x}`/`${NAME+x}`) -- all common defensive- + # scripting idioms for "reference a variable that might not be + # set." Bash's OTHER parameter-expansion operators that also + # evaluate to the empty string on an unset variable -- substring + # (`${NAME:0:5}`), prefix/suffix removal (`${NAME#x}`/ + # `${NAME%x}`), pattern substitution (`${NAME/x/y}`), and case + # modification (`${NAME^^}`) among others -- are NOT recognized: + # confirmed live these are not honest-accident-shaped the way the + # closed forms are (an ordinary script does not reach for prefix + # removal or case-folding just to guard against an unset + # variable), matching this file's own established convention for + # "exotic non-literal indirection" (issue #1375's own Non-goals + # section) rather than the near-zero-effort, ordinary-idiom bypass + # class the closed forms addressed. Confirmed live: this decoy is + # NOT silently allowed as if `checkout` were genuinely resolved -- + # the `git` occurrence is correctly treated as ambiguous and the + # command is simply never recognized as a checkout/restore + # invocation at all (checkout_restore_paths stays empty, matching + # the SAME disclosed-residual shape `V=checkout; git $V -- f.py` + # already carries), not a distinct or worse failure mode. + "git ${NEVERSET#x} checkout -- file.py", + "checkout-restore-exotic-parameter-expansion-decoy", + ), ] From 6a7a1dd8d09ebbe7a8b864d58fd58173b1ad86be Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 29 Aug 2026 11:02:17 +0000 Subject: [PATCH 08/46] fix(hooks): close a live checkout/restore bypass via line continuation A fresh, independent adversarial review of this PR's current head found a real, live-verified bypass: an ordinary backslash-newline line continuation before a checkout/restore path (a common style for wrapping a long git command) tokenized to a path with a literal leading newline baked in ('\nfile.py' instead of 'file.py'). The live git diff wrapper check then ran against that nonexistent path, found it clean, and silently allowed a real, dirty-file checkout through. Root cause: shlex's own posix-mode escape handling does not implement POSIX shell's backslash-newline line-joining rule -- it leaves a stray newline character embedded in the token (unquoted) or leaves both characters untouched (double-quoted), instead of removing the pair entirely the way real bash does. Fixed with a narrow, quote-aware preprocessing pass, _strip_line_ continuations, run before shlex ever sees the command: it tracks only single/double/unquoted state and removes exactly a backslash-newline pair when not inside single quotes, passing every other character through unchanged so shlex's own existing escape resolution still applies to everything else exactly as before. Verified live against real bash across unquoted, double-quoted, and single-quoted contexts, including the escaped-backslash-then-newline edge case where the second backslash must not be treated as a fresh escape-introducer. Also disclosed a second, non-blocking finding from the same review: a bare checkout/restore restores the working tree from the index, not HEAD, so a staged-with-no-further-edit path is a genuine no-op that this check still denies as if it would discard something. Confirmed directionally safe (over-denial only, never a missed real discard) and pinned with a regression test and an inline comment rather than special-cased, since the existing deny message's remedies already resolve it as a false alarm. Refs #1375. --- hooks/check-bash-safety.sh | 17 ++++ hooks/gitapex_check_bash_safety.py | 92 +++++++++++++++++++ hooks/test_gitapex_check_bash_safety.py | 47 ++++++++++ ...st_gitapex_check_bash_safety_properties.py | 70 ++++++++++++++ 4 files changed, 226 insertions(+) diff --git a/hooks/check-bash-safety.sh b/hooks/check-bash-safety.sh index ff4b024c..ad6ac481 100755 --- a/hooks/check-bash-safety.sh +++ b/hooks/check-bash-safety.sh @@ -237,6 +237,23 @@ if [ "$checkout_restore_paths_count" -gt 0 ]; then else diff_base="4b825dc642cb6eb9a060e54bf8d69288fbee4904" fi + # Disclosed, accepted residual (round-3 independent review, issue #1375): + # a bare `git checkout -- PATH` / `git restore PATH` restores the + # WORKING TREE from the INDEX, not from HEAD -- so a path that was + # `git add`-ed with no further unstaged edit (worktree == index, but + # index != HEAD) is a genuine no-op checkout/restore, yet this check + # diffs against HEAD/the empty tree and denies it as if it would + # discard something. Confirmed live: stage a change with no further + # edit, then `git checkout -- PATH` changes nothing on disk, but this + # diff_base comparison still reports a difference. Deliberately left + # as-is rather than special-cased per flag combination (`--staged` + # alone already skips this check entirely below, since unstaging alone + # never discards file content) -- the failure direction is safe + # (over-denial only, matching every other explicit-source variant this + # check already treats the same conservative way; it never under-denies + # a real discard), and the existing deny message's `git checkout -m --` + # / `git add` remedies still resolve it as a false alarm the caller can + # work around. # Fed via process substitution (`< <(...)`), not a pipe # (`... | while read`) -- bash runs a pipe's right-hand side in a # subshell, where `deny`'s own `exit 2` would only exit that subshell, diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index e1f3a7bc..3c2eebde 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -872,6 +872,94 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b return None, is_git_push, tuple(checkout_restore_paths) +def _strip_line_continuations(command: str) -> str: + """POSIX shell removes a backslash immediately followed by a newline + ("\\") entirely, wherever backslash is an active escape + character -- unquoted and inside double quotes -- joining the two + physical lines into one logical line with nothing left behind + (confirmed live: `echo \\` + newline + `text` runs as `echo text`, + one argument, no embedded newline). Inside single quotes, backslash + has no special meaning at all, so a literal backslash-newline pair + there stays exactly as written (confirmed live). + + `shlex`'s own posix-mode escape handling does not implement this + line-joining rule: for an UNQUOTED backslash-newline it strips only + the backslash and leaves the newline embedded in the resulting token; + for a DOUBLE-QUOTED one it leaves both characters untouched. So a + completely ordinary line-continued git command -- `git checkout -- + \\` + newline + `file.py`, an everyday style for wrapping a long + command -- tokenized to a path token with a leading literal newline + baked in (`'\\nfile.py'`) instead of the real path (`'file.py'`). + `_resolve_path_tokens` then resolved that literal (non-dynamic) + token as-is, and the wrapper's live `git diff` check ran against a + nonexistent path and reported clean, silently allowing a real, + dirty-file checkout through. A live, verified bypass of the entire + checkout/restore guard for ordinary line-wrapping, not exotic + obfuscation. Found by independent adversarial review of the + checkout/restore feature itself (round 3, issue #1375). + + This is a narrow, single-purpose preprocessing pass run BEFORE + `shlex` ever sees the command: it tracks only single-vs-double-vs- + unquoted state (the same quote state `shlex` itself already computes + downstream) and removes exactly a backslash immediately followed by + a newline, only when not inside single quotes. Every other + character -- including every other backslash-escape sequence -- is + passed through completely UNCHANGED (byte-for-byte identical to the + input) for `shlex`'s own existing, already-tested escape resolution + to still handle exactly as before; the only possible divergence from + pure identity-passthrough is the removed backslash-newline pairs. + + An escape pair is always consumed two source characters at a time + (never one), so a `\\\\` (an escaped literal backslash) does not + leave its second backslash dangling as a fresh, wrongly-re-examined + escape-introducer for whatever follows -- confirmed live this + matters: `"\\\\` + newline + `b"` keeps the newline (the second + backslash was already spent escaping the first, so the newline that + follows is a plain literal character), while `"\\` + newline + `b"` + (a single leading backslash) removes it.""" + result: list[str] = [] + in_single = False + in_double = False + i = 0 + n = len(command) + while i < n: + ch = command[i] + if in_single: + result.append(ch) + if ch == "'": + in_single = False + i += 1 + continue + if ch == "\\" and i + 1 < n: + nxt = command[i + 1] + if nxt == "\n": + i += 2 + continue + result.append(ch) + result.append(nxt) + i += 2 + continue + if in_double: + if ch == '"': + in_double = False + result.append(ch) + i += 1 + continue + if ch == "'": + in_single = True + result.append(ch) + i += 1 + continue + if ch == '"': + in_double = True + result.append(ch) + i += 1 + continue + result.append(ch) + i += 1 + return "".join(result) + + def tokenize(command: str) -> list[str]: """Raises TokenizeError on anything shlex cannot parse (e.g. an unbalanced quote) -- the caller must fail closed on that, the same @@ -882,6 +970,9 @@ def tokenize(command: str) -> list[str]: content` against these still-unfolded tokens, which needs each span's own inner tokens still separable. + Runs `_strip_line_continuations` first -- see that function's own + docstring for the live-verified backslash-newline bypass it closes. + An UNQUOTED newline is one of `segment_tokens`'s own documented segment boundaries (it is a member of `_SINGLE_OPS`) -- but shlex's own default `whitespace` set includes `\\n`, so a bare newline was @@ -906,6 +997,7 @@ def tokenize(command: str) -> list[str]: enclosing token, since shlex's own quote handling runs before punctuation-splitting) nor any command with no literal newline in it at all (the entire pre-existing test suite's own command strings).""" + command = _strip_line_continuations(command) try: lexer = shlex.shlex(command, posix=True, punctuation_chars="();<>|&\n") lexer.whitespace_split = True diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 8c8452ba..b4736063 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1353,6 +1353,23 @@ def test_checkout_denied_when_target_has_uncommitted_changes(tmp_path: Path) -> assert "git stash" in payload["systemMessage"] +def test_checkout_denied_when_the_command_uses_an_ordinary_line_continuation(tmp_path: Path) -> None: + """CRITICAL regression pin (round-3 independent review, issue #1375). + An everyday line-wrapping style for a long git command -- a trailing + backslash before the newline -- must still resolve to the real path + (`f.py`), not a path with a literal leading newline baked in that the + live `git diff` check would silently run against a nonexistent path + and allow through.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir) + file_path.write_text("hello\ndirty\n") + result = run("git checkout -- \\\nf.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert payload["hookSpecificOutput"]["permissionDecision"] == "deny" + assert "f.py" in payload["systemMessage"] + + def test_checkout_denied_from_a_subdirectory_when_target_has_uncommitted_changes(tmp_path: Path) -> None: """The near-miss's own exact shape (issue #1375, issue #1128 repair 4): replayed from a SUBDIRECTORY of the repo, not just the repo root -- @@ -1379,6 +1396,36 @@ def test_checkout_allowed_when_target_is_clean(tmp_path: Path) -> None: assert result.stderr == "" +def test_checkout_denied_for_a_staged_no_op_pins_the_disclosed_over_denial(tmp_path: Path) -> None: + """Disclosed, accepted residual (round-3 independent review, issue + #1375), pinned rather than left silently uncovered: `git checkout -- + PATH` restores the working tree from the INDEX, not HEAD, so staging a + change with no further unstaged edit (worktree == index, index != + HEAD) makes the checkout a genuine no-op on disk. This check still + diffs against HEAD and denies it -- over-denial only, the safe + direction, never a missed real discard. If this check is ever changed + to diff against the index for the non-`--staged` case, this test's + own assertion must flip to `returncode == 0` alongside it.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir) + file_path.write_text("hello\nstaged\n") + _git(repo_dir, "add", "f.py") + result = run("git checkout -- f.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert payload["hookSpecificOutput"]["permissionDecision"] == "deny" + before = file_path.read_text() + subprocess.run( + ["git", "checkout", "--", "f.py"], + cwd=str(repo_dir), + check=True, + capture_output=True, + text=True, + timeout=10, + ) + assert file_path.read_text() == before, "real git checkout is a no-op here, confirming the denial was spurious" + + def test_checkout_dot_denied_when_a_tracked_file_is_dirty(tmp_path: Path) -> None: repo_dir = tmp_path / "repo" file_path = _init_repo_with_committed_file(repo_dir) diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 160bc91e..21412c44 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3192,6 +3192,76 @@ def test_shlex_default_punctuation_chars_still_matches_the_hardcoded_extension() assert default_lexer.punctuation_chars == "();<>|&" +def test_strip_line_continuations_removes_an_unquoted_backslash_newline() -> None: + """CRITICAL regression pin (round-3 independent review). An ordinary + line-continued command -- backslash immediately followed by a newline, + outside any quoting -- must vanish entirely, exactly as real bash + resolves it, joining the two physical lines with nothing left behind.""" + assert checker._strip_line_continuations("git checkout -- \\\nfile.py") == "git checkout -- file.py" + + +def test_strip_line_continuations_removes_a_double_quoted_backslash_newline() -> None: + """Real bash also removes a backslash-newline pair INSIDE double + quotes (backslash retains its escaping meaning there), unlike shlex's + own posix-mode escape handling, which left both characters untouched.""" + assert checker._strip_line_continuations('echo "a\\\nb"') == 'echo "ab"' + + +def test_strip_line_continuations_preserves_a_single_quoted_backslash_newline() -> None: + """Inside single quotes, backslash has no special meaning at all, so a + literal backslash-newline pair there must stay exactly as written -- + confirmed live real bash does not join these two lines.""" + assert checker._strip_line_continuations("echo 'a\\\nb'") == "echo 'a\\\nb'" + + +def test_strip_line_continuations_does_not_double_consume_an_escaped_backslash() -> None: + """An escaped literal backslash (`\\\\`) must consume its own pair + atomically so the second backslash is never re-examined as a fresh, + wrongly-applied escape-introducer for the newline that follows it -- + confirmed live real bash keeps this newline (the second backslash was + already spent escaping the first).""" + assert checker._strip_line_continuations('echo "a\\\\\nb"') == 'echo "a\\\\\nb"' + + +def test_strip_line_continuations_preserves_a_raw_quoted_newline() -> None: + """A genuine embedded newline inside a quoted string, with no + preceding backslash at all, is real string content, not a line + continuation, and must never be stripped.""" + assert checker._strip_line_continuations('echo "line1\nline2"') == 'echo "line1\nline2"' + + +@_PROPERTIES +@given(text=st.text(alphabet=st.characters(blacklist_characters="\\'\"\n"), max_size=40)) +def test_strip_line_continuations_is_a_no_op_without_backslash_or_quote(text: str) -> None: + """Property: with no backslash, quote, or newline in the input at + all, `_strip_line_continuations` cannot find anything to remove, so it + must return the input byte-for-byte unchanged.""" + assert checker._strip_line_continuations(text) == text + + +@_PROPERTIES +@given(text=st.text(alphabet="ab\\'\"\n", max_size=12)) +def test_strip_line_continuations_is_idempotent(text: str) -> None: + """Property: running the pass twice must equal running it once -- once + every unescaped, non-single-quoted backslash-newline pair is removed, + a second pass over the result finds nothing further to remove.""" + once = checker._strip_line_continuations(text) + twice = checker._strip_line_continuations(once) + assert once == twice + + +def test_classify_denies_a_line_continued_checkout_path_that_used_to_bypass() -> None: + """End-to-end regression pin for the round-3 independent-review + finding: an ordinary `\\`-then-newline-wrapped `git checkout -- + file.py` must resolve to the real path (`file.py`), not a path with a + literal leading newline baked in (`'\\nfile.py'`, which the live `git + diff` wrapper check would silently run against a nonexistent path and + allow through).""" + verdict = checker.classify("git checkout -- \\\nfile.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("file.py",) + + @_PROPERTIES @given(command_paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_paths: list[str]) -> None: From aed21be1ca8fd2fd244db09372e69c8bdd253498 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 11:23:12 +0000 Subject: [PATCH 09/46] fix(hooks): stop treating -b/-B/--orphan's own value as a checkout path A fresh, independent adversarial review of this PR's current head found a second real, live-verified bypass: `-b`/`-B`/`--orphan` is git's own branch-creation/reset mode for checkout, mutually exclusive with every pathspec-checkout mode, but `-b`/`-B` take the immediately following token as their own new-branch-NAME value, which does not start with `-`. Sub-case (b)'s dash-prefix positional filter swept that value (and a start-point after it) into checkout_restore_paths as if they were file paths, so `git checkout -f -b newbranch other` reported ('newbranch', 'other') -- neither the real at-risk file. Live-verified end-to-end in a scratch repo: dirtied a tracked file with no relation to either token, ran the real command, and confirmed the change was silently discarded while the wrapper's live git diff check against the two wrong (nonexistent) paths found "clean" and allowed the command through -- a false safety claim, not merely an unchecked case. Fixed by checking for -b/-B/--orphan first in _git_checkout_paths, before any sub-case dispatch, and folding it into the same honest, no-claim Non-goal the bare `git checkout SOMENAME` case already carries: empty checkout_restore_paths, matching the risk profile this exact command already had before this feature existed (git's own built-in switch-protection, minus whatever -f already bypasses), rather than a confident, wrong claim that specific paths were checked and clean. Soundly extracting the real at-risk set for a branch creation/reset would need to reproduce git's own internal "would this overwrite any dirty tracked file" logic, out of a pure classifier's reach -- the same reasoning that already accepts the bare-SOMENAME case as a Non-goal. Updated one pre-existing test whose own fixture command happened to be exactly this shape (`git checkout -b newbranch master`, used to pin the unrelated newline-segment-boundary fix) to use plain positionals instead, preserving its original intent without colliding with this fix. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 43 ++++++++++++-- hooks/test_gitapex_check_bash_safety.py | 23 ++++++++ ...st_gitapex_check_bash_safety_properties.py | 59 ++++++++++++++++++- 3 files changed, 116 insertions(+), 9 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 3c2eebde..f4d24a80 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -2647,6 +2647,7 @@ def _is_git_push_segment(seg: list[str], name_to_raw_value: dict[str, str]) -> b "--ignore-skip-worktree-bits", } _RESTORE_VALUE_FLAGS = {"--source", "-s", "--conflict"} +_CHECKOUT_BRANCH_CREATION_FLAGS = {"-b", "-B", "--orphan"} def _resolve_path_tokens(tokens: list[str], name_to_raw_value: dict[str, str]) -> tuple[str | None, tuple[str, ...]]: @@ -2717,11 +2718,11 @@ def _git_checkout_paths( (b) No `--`, 2+ non-flag-shaped positional tokens -- confirmed live that `git checkout no-such-ref no-such-file` (two unresolvable positionals, no `--`) reports a pathspec error for BOTH, meaning - whenever real git is given 2+ positionals with no `--`, every - position past the first is a pathspec under every resolution git - can take. Over-including a token that also happens to be a valid - ref name just checks a path that likely does not exist, which is - harmless. + whenever real git is given 2+ positionals with no `--` AND no + `-b`/`-B`/`--orphan` (see below), every position past the first is + a pathspec under every resolution git can take. Over-including a + token that also happens to be a valid ref name just checks a path + that likely does not exist, which is harmless. (c) No `--`, exactly one positional token, and it is the literal `.` or `..` -- both are syntactically invalid git ref names (confirmed live: `git check-ref-format --branch .`/`--branch ..` both fail, @@ -2732,7 +2733,37 @@ def _git_checkout_paths( Bare `git checkout SOMENAME` (single positional, not `.`/`..`, no `--`) is a deliberate Non-goal: SOMENAME might be a branch/ref name or a path, and disambiguating soundly needs a live ref-existence lookup - this pure classifier does not perform.""" + this pure classifier does not perform. + + `-b`/`-B`/`--orphan` (git's own branch-creation/reset mode, mutually + exclusive with every pathspec-checkout mode above per `git checkout + -h`'s own synopsis) is checked FIRST, before any sub-case above, and + folded into that same Non-goal -- CRITICAL bug found by independent + adversarial review (round 4, issue #1375) and independently + reproduced live: `-b`/`-B` take the immediately following token as + their own new-branch-NAME value, which does not start with `-`, so + sub-case (b)'s own dash-prefix positional filter swept a value like + `git checkout -f -b newbranch other` into `checkout_restore_paths = + ('newbranch', 'other')` -- neither of which is the actual at-risk + file -- and the wrapper's live `git diff --quiet` check against those + two nonexistent paths found "clean" and silently ALLOWED a real, + forced branch switch that discarded an uncommitted change to an + entirely different, unchecked file. Worse than the already-accepted + bare-SOMENAME Non-goal above: that one makes NO claim at all (falls + through with an empty `checkout_restore_paths`, the same as if this + classifier had never seen the command, honestly matching real git's + own built-in switch-protection minus whatever `-f` already bypasses); + sub-case (b)'s old behavior here instead made a CONFIDENT, WRONG claim + that specific paths were checked and clean. Folding this case into the + Non-goal (rather than a live-git-lookup-free sound extraction, which + would need to reproduce git's own internal "would this branch switch + overwrite ANY dirty tracked file in the whole working tree" logic -- + out of a pure classifier's reach) restores the honest, no-claim + behavior and removes the false-confidence gap; it does not newly + regress anything `-f`/`-b` could already do to an unguarded working + tree before this classifier existed at all.""" + if any(tok in _CHECKOUT_BRANCH_CREATION_FLAGS or tok.startswith("--orphan=") for tok in tokens_after): + return None, () if "--" in tokens_after: after = tokens_after[tokens_after.index("--") + 1 :] if not after: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index b4736063..1dd3a498 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -394,6 +394,29 @@ def assert_allowed(command: str) -> None: "git ${NEVERSET#x} checkout -- file.py", "checkout-restore-exotic-parameter-expansion-decoy", ), + ( + # Found live by independent adversarial review (round 4, issue + # #1375): `-b`/`-B`/`--orphan` is git's own branch-creation mode, + # mutually exclusive with every pathspec-checkout mode -- but + # `-b`/`-B` take the immediately following token as their own + # new-branch-NAME value, which does not start with `-`, so this + # command used to sweep "newbranch"/"other" into + # `checkout_restore_paths` as if they were file paths instead of + # the actual at-risk file. Live-verified before the fix: the + # wrapper's check against those two nonexistent paths found + # "clean" and allowed a real, forced branch switch through that + # silently discarded an uncommitted change elsewhere. Now folded + # into the same honest, no-claim Non-goal `git checkout SOMENAME` + # already carries (empty `checkout_restore_paths`, not a false + # claim) -- disambiguating a branch-creation/reset's own working- + # tree impact soundly would need to reproduce git's internal + # "would this overwrite ANY dirty tracked file" logic, out of a + # pure classifier's reach, the same reasoning that already accepts + # the bare-SOMENAME case as a Non-goal rather than a sound + # extraction. + "git checkout -f -b newbranch other", + "checkout-branch-creation-flag-non-goal", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 21412c44..75142c3e 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3175,10 +3175,14 @@ def test_classify_does_not_leak_a_later_lines_token_into_an_earlier_checkout() - """End-to-end regression pin for the newline finding: an ordinary two-line script with `git checkout` on the first line and an unrelated `$`-containing token on the second line must classify the - checkout using only its own line's tokens.""" - verdict = checker.classify('git checkout -b newbranch master\necho "exit=$?"') + checkout using only its own line's tokens. Deliberately avoids `-b` + (which the round-4 branch-creation-flag fix folds into the Non-goal + regardless of what follows on either line, no longer exercising + sub-case (b)'s own 2-positional path extraction this test targets) -- + two ordinary non-flag positionals exercise the identical sub-case.""" + verdict = checker.classify('git checkout a.py b.py\necho "exit=$?"') assert verdict.deny is False - assert verdict.checkout_restore_paths == ("newbranch", "master") + assert verdict.checkout_restore_paths == ("a.py", "b.py") def test_shlex_default_punctuation_chars_still_matches_the_hardcoded_extension() -> None: @@ -3262,6 +3266,55 @@ def test_classify_denies_a_line_continued_checkout_path_that_used_to_bypass() -> assert verdict.checkout_restore_paths == ("file.py",) +@pytest.mark.parametrize("flag", ["-b", "-B", "--orphan"]) +def test_git_checkout_paths_folds_branch_creation_flags_into_the_non_goal(flag: str) -> None: + """CRITICAL regression pin (round-4 independent review, issue #1375). + `-b`/`-B`/`--orphan` take the immediately following token as their own + new-branch-NAME value, which does not start with `-` -- sub-case (b)'s + dash-prefix positional filter used to sweep that value (and a + start-point after it) into `checkout_restore_paths` as if they were + file paths, so `git checkout -f -b newbranch other` reported + `('newbranch', 'other')` -- neither the real at-risk file -- and the + wrapper's live check against those two nonexistent paths found "clean" + and silently allowed a real, forced branch switch that discarded an + uncommitted change elsewhere. Live-verified end-to-end that real git + discards the change while the old code reported this as checked-safe. + Must now fold into the same honest, no-claim Non-goal bare `git + checkout SOMENAME` already carries -- empty paths, not a false claim.""" + reason, paths = checker._git_checkout_paths([flag, "newbranch", "other"], {}) + assert reason is None + assert paths == () + + +def test_git_checkout_paths_branch_creation_flag_wins_even_with_a_double_dash() -> None: + """`-b`/`-B`/`--orphan` is git's own branch-creation mode, mutually + exclusive with every pathspec-checkout mode (per `git checkout -h`'s + own synopsis) -- the Non-goal fold must fire before sub-case (a)'s own + `--`-present branch is ever reached, not only when `--` is absent.""" + reason, paths = checker._git_checkout_paths(["-b", "newbranch", "--", "file.py"], {}) + assert reason is None + assert paths == () + + +def test_git_checkout_paths_still_extracts_a_real_path_without_a_branch_creation_flag() -> None: + """No regression from the branch-creation fold: an ordinary two- + positional pathspec checkout with no `-b`/`-B`/`--orphan` present is + unaffected.""" + reason, paths = checker._git_checkout_paths(["a.py", "b.py"], {}) + assert reason is None + assert paths == ("a.py", "b.py") + + +def test_classify_no_longer_falsely_claims_safety_for_a_forced_branch_creation() -> None: + """End-to-end regression pin for the round-4 independent-review + finding: `git checkout -f -b newbranch other` must resolve to an empty + `checkout_restore_paths` (the honest Non-goal), never a claim naming + the branch name/start-point as though they were the checked paths.""" + verdict = checker.classify("git checkout -f -b newbranch other") + assert verdict.deny is False + assert verdict.checkout_restore_paths == () + + @_PROPERTIES @given(command_paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_paths: list[str]) -> None: From 86335c3d8800559d342aea1f4b2b33644842bee9 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 11:38:13 +0000 Subject: [PATCH 10/46] fix(hooks): deny checkout --pathspec-from-file, matching restore's own rule A fresh, independent adversarial review of this PR's current head found a third real, live-verified bypass: _git_restore_paths already hard- denies --pathspec-from-file/--pathspec-file-nul ("paths come from a file this classifier cannot inspect"), but _git_checkout_paths -- despite real git accepting the identical flag pair on checkout, not just restore -- never recognized it at all. A single positional after it (the control file's own name) fell through to the bare-SOMENAME Non-goal, which is the wrong treatment here: that Non-goal is honest for an ambiguous ref/path, but --pathspec-from-file's value names a FILE CONTAINING the real pathspecs, which is exactly the opaque-path threat this whole feature exists to close, not a case to silently wave through. Live-verified end-to-end: with a tracked file listed in the control file dirtied, the wrapper allowed `git checkout --pathspec-from-file files.txt` unconditionally (no check performed at all), and the real command silently discarded the change. Fixed by adding the same explicit deny _git_restore_paths already carries for this flag pair, checked right after the round-4 branch- creation-flag fold and before any positional-based sub-case dispatch. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 36 ++++++++++++++++++- hooks/test_gitapex_check_bash_safety.py | 20 +++++++++++ ...st_gitapex_check_bash_safety_properties.py | 25 +++++++++++++ 3 files changed, 80 insertions(+), 1 deletion(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index f4d24a80..00d66875 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -2761,9 +2761,43 @@ def _git_checkout_paths( out of a pure classifier's reach) restores the honest, no-claim behavior and removes the false-confidence gap; it does not newly regress anything `-f`/`-b` could already do to an unguarded working - tree before this classifier existed at all.""" + tree before this classifier existed at all. + + `--pathspec-from-file`/`--pathspec-file-nul` (real git accepts both on + `checkout`, not just `restore`) is checked next and DENIES outright -- + CRITICAL bug found by independent adversarial review (round 5, issue + #1375) and independently reproduced live: `_git_restore_paths` already + hard-denies this exact flag pair ("paths come from a file this + classifier cannot inspect"), but `_git_checkout_paths` never + recognized it at all, so `git checkout --pathspec-from-file + files.txt` (a single positional, `files.txt`, itself not `.`/`..`) + fell all the way through to the bare-SOMENAME Non-goal above -- an + HONEST no-claim shape for an ordinary ambiguous ref/path, but not for + a flag whose own value-consumption is a FILE CONTAINING THE REAL + PATHSPECS this classifier cannot read. Live-verified: with a tracked + file listed in that control file dirtied, the wrapper allowed the + command (exit 0, no check performed) and the real `git checkout + --pathspec-from-file` silently discarded the change. Denying here, + matching restore's own established treatment, rather than folding + into the Non-goal: unlike the `-b`/`-B` case above (where an + unresolvable "would this overwrite anything" question is inherent to + branch switching itself, matching git's own already-imperfect native + protection), a pathspec-from-file's paths are knowable in principle -- + this classifier simply cannot read the named file -- so silently + granting no-claim safety here would under-serve the exact opaque-path + threat model this whole feature exists to close, not merely decline + to extend coverage.""" if any(tok in _CHECKOUT_BRANCH_CREATION_FLAGS or tok.startswith("--orphan=") for tok in tokens_after): return None, () + if any( + tok == "--pathspec-from-file" or tok.startswith("--pathspec-from-file=") or tok == "--pathspec-file-nul" + for tok in tokens_after + ): + return ( + "a 'git checkout --pathspec-from-file'/'--pathspec-file-nul' flag reads paths from a file this " + "classifier cannot inspect, so this is denied outright", + (), + ) if "--" in tokens_after: after = tokens_after[tokens_after.index("--") + 1 :] if not after: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 1dd3a498..378b887c 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1393,6 +1393,26 @@ def test_checkout_denied_when_the_command_uses_an_ordinary_line_continuation(tmp assert "f.py" in payload["systemMessage"] +def test_checkout_denied_for_pathspec_from_file(tmp_path: Path) -> None: + """CRITICAL regression pin (round-5 independent review, issue #1375). + `_git_restore_paths` already hard-denied `--pathspec-from-file`, but + `_git_checkout_paths` never recognized it -- a single positional after + it fell through to the honest bare-SOMENAME Non-goal, which is the + WRONG treatment for a flag whose value is a file naming the real + pathspecs. Live-verified before the fix: with a tracked file listed + in that control file dirtied, the wrapper allowed the command + unconditionally and the real checkout silently discarded the change.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir) + file_path.write_text("hello\ndirty\n") + (repo_dir / "files.txt").write_text("f.py\n") + result = run("git checkout --pathspec-from-file files.txt", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert payload["hookSpecificOutput"]["permissionDecision"] == "deny" + assert "pathspec-from-file" in payload["systemMessage"] + + def test_checkout_denied_from_a_subdirectory_when_target_has_uncommitted_changes(tmp_path: Path) -> None: """The near-miss's own exact shape (issue #1375, issue #1128 repair 4): replayed from a SUBDIRECTORY of the repo, not just the repo root -- diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 75142c3e..9b064d27 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3315,6 +3315,31 @@ def test_classify_no_longer_falsely_claims_safety_for_a_forced_branch_creation() assert verdict.checkout_restore_paths == () +@_PROPERTIES +@given(flag=st.sampled_from(["--pathspec-from-file=list.txt", "--pathspec-from-file", "--pathspec-file-nul"])) +def test_git_checkout_paths_denies_pathspec_from_file(flag: str) -> None: + """CRITICAL regression pin (round-5 independent review, issue #1375). + `_git_restore_paths` already hard-denies this exact flag pair ("paths + come from a file this classifier cannot inspect"), but + `_git_checkout_paths` never recognized it at all -- a single + positional after it fell through to the honest bare-SOMENAME Non-goal, + which is the WRONG treatment for a flag whose own value-consumption is + a file containing the real pathspecs, not an ambiguous ref/path. + Live-verified end-to-end that this silently discarded a dirty tracked + file listed in the control file.""" + reason, resolved = checker._git_checkout_paths([flag, "files.txt"], {}) + assert reason is not None + assert resolved == () + + +def test_classify_denies_checkout_pathspec_from_file() -> None: + """End-to-end regression pin for the round-5 finding at the + `classify()` level, mirroring the already-existing restore-side pin.""" + verdict = checker.classify("git checkout --pathspec-from-file files.txt") + assert verdict.deny is True + assert "pathspec-from-file" in verdict.reason + + @_PROPERTIES @given(command_paths=st.lists(_PATH_TOKENS, min_size=1, max_size=3)) def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_paths: list[str]) -> None: From dfebaa218794dc61aeed44ec3d6b76c87664edbc Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 12:11:14 +0000 Subject: [PATCH 11/46] fix(hooks): keep a line-continued comment recognized after the merge A fresh, independent adversarial review of this PR's merge-conflict resolution (round 6) found a real bug in _strip_comments's own boundary tracking, present since issue #1350's already-merged version but with no observable consequence there (main had no checkout/restore feature for a leaked comment token to reach) -- this merge is what first makes it security-relevant. _strip_comments unconditionally cleared its own word-boundary flag after consuming any backslash-escaped pair, including a genuine \ line continuation. A continuation deletes to nothing once _strip_line_continuations runs afterward, so the character right after it should see whatever boundary status held immediately before the backslash -- not a forced non-boundary. The bug meant a "#" comment sitting on a continued line was never recognized as a comment. Live-verified end-to-end: `git checkout -- clean.py \` + newline + `# TODO revisit auth.py later` swept "auth.py" (a name that only appears in the comment text) into checkout_restore_paths as a phantom candidate, and the wrapper denied an entirely safe checkout with a message pointing at a file the command never referenced. Over-denial only, never a missed real discard, but a confusing false positive for a completely ordinary bash idiom (an inline comment on a wrapped command). Fixed by not clearing the boundary flag specifically when the escaped character is a newline, in both the unquoted and double-quoted backslash-handling branches (the double-quoted one has no currently observable effect, since exiting a quote already forces the flag false regardless, but is fixed too for consistency with the documented invariant and to avoid a future refactor silently making it live). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 37 +++++++++++++--- hooks/test_gitapex_check_bash_safety.py | 26 ++++++++++++ ...st_gitapex_check_bash_safety_properties.py | 42 +++++++++++++++++++ 3 files changed, 100 insertions(+), 5 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 1106a342..98f4831b 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -959,7 +959,30 @@ def _strip_comments(command: str) -> str: backslash inert) -- a comment always ends at the very next raw newline in COMMAND, full stop, which is exactly what searching for the next raw `\\n` (rather than delegating to `_strip_line_continuations` - first) gives here.""" + first) gives here. + + A genuine line continuation (`\\` immediately followed by a raw + newline) is the ONE backslash-pair shape that does NOT clear + AT_BOUNDARY -- CRITICAL bug found by independent adversarial review + (round 6, issue #1375, during this PR's own merge with issue #1350's + already-merged `_strip_comments`): a continuation vanishes with + NOTHING left behind once `_strip_line_continuations` runs afterward + (this function only passes the pair through unchanged; it does not + itself delete it), so the boundary status right after a continuation + must be whatever it was right BEFORE the backslash, exactly as if the + continuation were not there at all -- every OTHER escaped pair (an + escaped literal character that genuinely survives into the output, + like `\\#`) correctly still clears it, since that character is real, + non-boundary word content. Confirmed live this was a real, security- + relevant leak once combined with issue #1375's own checkout/restore + feature: `git checkout -- clean.py \\` + newline + `# TODO revisit + auth.py later` used to tokenize with `#` never recognized as a + comment-starter (AT_BOUNDARY wrongly cleared by the continuation + pair), sweeping `auth.py` (an unrelated filename that merely happens + to appear in the comment text) into `checkout_restore_paths` as a + phantom candidate, and denying an entirely safe checkout with a + misleading message naming a file the command never referenced. Only + an over-denial (never a missed real discard), but a confusing one.""" out: list[str] = [] in_single_quote = False in_double_quote = False @@ -977,10 +1000,12 @@ def _strip_comments(command: str) -> str: continue if in_double_quote: if char == "\\" and i + 1 < n: + nxt = command[i + 1] out.append(char) - out.append(command[i + 1]) + out.append(nxt) i += 2 - at_boundary = False + if nxt != "\n": + at_boundary = False continue out.append(char) if char == '"': @@ -989,10 +1014,12 @@ def _strip_comments(command: str) -> str: i += 1 continue if char == "\\" and i + 1 < n: + nxt = command[i + 1] out.append(char) - out.append(command[i + 1]) + out.append(nxt) i += 2 - at_boundary = False + if nxt != "\n": + at_boundary = False continue if char == "'": in_single_quote = True diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 378b887c..6a71c9f0 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1413,6 +1413,32 @@ def test_checkout_denied_for_pathspec_from_file(tmp_path: Path) -> None: assert "pathspec-from-file" in payload["systemMessage"] +def test_checkout_allowed_when_a_comment_after_a_line_continuation_names_an_unrelated_dirty_file( + tmp_path: Path, +) -> None: + """CRITICAL regression pin (round-6 independent review, issue #1375). + `_strip_comments` used to wrongly clear its own boundary status across + a genuine line continuation, so a `#`-comment sitting on the continued + line was never recognized as a comment -- its text (here naming an + unrelated, genuinely dirty file) got swept into `checkout_restore_paths` + as a phantom candidate and produced a misleading deny. The real + checkout target (`f.py`) is untouched; `auth.py` is dirty but never + referenced by the actual command, only by the comment text.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + (repo_dir / "auth.py").write_text("hello\n") + _git(repo_dir, "add", "auth.py") + _git(repo_dir, "commit", "-q", "-m", "add auth.py") + (repo_dir / "auth.py").write_text("hello\ndirty\n") + result = run( + "git checkout -- f.py \\\n# TODO revisit auth.py later\n", + payload_cwd=str(repo_dir), + ) + assert result.returncode == 0, f"stderr={result.stderr!r}" + assert result.stdout == "" + assert result.stderr == "" + + def test_checkout_denied_from_a_subdirectory_when_target_has_uncommitted_changes(tmp_path: Path) -> None: """The near-miss's own exact shape (issue #1375, issue #1128 repair 4): replayed from a SUBDIRECTORY of the repo, not just the repo root -- diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 9b064d27..5f5276b0 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3266,6 +3266,48 @@ def test_classify_denies_a_line_continued_checkout_path_that_used_to_bypass() -> assert verdict.checkout_restore_paths == ("file.py",) +def test_strip_comments_preserves_boundary_status_across_a_line_continuation() -> None: + """CRITICAL regression pin (round-6 independent review, issue #1375). + A genuine line continuation vanishes with nothing left behind once + `_strip_line_continuations` runs afterward -- `_strip_comments` only + passes the pair through unchanged, so the boundary status right after + it must be whatever it was right BEFORE the backslash, not forced to + False the way every other escaped pair correctly is. A `#` right + after a continuation must still start a comment.""" + result = checker._strip_comments("echo a \\\n#comment\necho b") + assert result == "echo a \\\n\necho b" + + +def test_strip_comments_passes_through_a_double_quoted_line_continuation_unchanged() -> None: + """The round-6 fix's own double-quoted branch: a continuation pair + INSIDE an open double-quoted string is passed through unchanged (no + comment can start there regardless of boundary status, since `#` is + only ever checked in the top-level unquoted branch), covering the + `nxt == "\\n"` skip-path this function's double-quoted backslash + handling shares with its unquoted twin.""" + result = checker._strip_comments('echo "a \\\nb"') + assert result == 'echo "a \\\nb"' + + +def test_strip_comments_still_clears_boundary_for_a_non_continuation_escape() -> None: + """No regression from the round-6 fix: an escaped, non-newline + character is still real word content, not a boundary -- `\\#` right + after it must NOT be read as a comment-starter.""" + result = checker._strip_comments("echo a\\x#notacomment") + assert result == "echo a\\x#notacomment" + + +def test_classify_no_longer_leaks_a_comment_past_a_line_continuation() -> None: + """End-to-end regression pin for the round-6 independent-review + finding: a `#`-comment sitting on the continued line right after a + `\\` must be recognized and stripped, not swept into + `checkout_restore_paths` as a phantom path candidate that could name + an unrelated, genuinely dirty file and produce a misleading deny.""" + verdict = checker.classify("git checkout -- clean.py \\\n# TODO revisit auth.py later\n") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("clean.py",) + + @pytest.mark.parametrize("flag", ["-b", "-B", "--orphan"]) def test_git_checkout_paths_folds_branch_creation_flags_into_the_non_goal(flag: str) -> None: """CRITICAL regression pin (round-4 independent review, issue #1375). From 69c15c9609d43c96938d0042d162704442b8447a Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 12:41:24 +0000 Subject: [PATCH 12/46] fix(hooks): recognize a comment inside a $(...) nested in double quotes A fresh, independent adversarial review of this PR's current head (round 7) found a critical, full-classifier-bypass bug, not limited to checkout/restore: _strip_comments treated everything inside an open double-quoted string as opaque literal text, with no comment recognition at all. Correct for genuine literal content ("a#b" really is one literal word in real bash), but wrong for a $(...) embedded inside that double-quoted string -- real bash recursively re-enters ordinary, comment-aware command parsing for a substitution's own content regardless of what quote encloses the $( that opened it, so a "#" inside it does start a real comment (confirmed live: a ")" inside a #-comment inside "$(...)" does not end the substitution). Left unstripped, that comment's own embedded ")" survived into shlex's dequoted token, where _find_fused_command_substitution's paren-depth counter (comment- and quote-blind by design) mistook it for the substitution's real closing paren, silently truncating everything after that point -- including a genuine, undisguised git checkout on the next physical line -- from all classification, not merely the checkout/restore rule. Live-verified, real, silent data loss: `x="$(echo hi #comment with paren ) here` + newline + `git checkout -- dirty.py)"` ran the embedded checkout for real and discarded an uncommitted change, while classify() reported deny=False with an EMPTY checkout_restore_paths -- a confident, wrong "nothing to see here" rather than an honest non-goal. Fixed by teaching _strip_comments's double-quote handling to recognize a nested $(...) and delegate to a new recursive helper, _consume_command_substitution_content, that processes the substitution's own content the way real bash does: comments live again, and any further nested '/"/$( gets its own independent, mutually-recursive handling (via a second new helper, _consume_double_quoted_content, also now used for ordinary top-level double-quoted strings). The analogous decoy built from a literal ")" inside a nested QUOTED span (rather than a comment) needed no fix and was checked live: any balanced quoted span containing a literal ")" necessarily leaves an odd, unbalanced quote count in text naively truncated partway through it, which already trips tokenize()'s own TokenizeError fail-closed path -- only a comment can hide an unbalanced ")" without requiring an unbalanced quote in the truncated prefix, which is why this fix is scoped to comment-handling rather than a general rewrite of the paren-depth counter. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 184 ++++++++++++++++-- hooks/test_gitapex_check_bash_safety.py | 27 +++ ...st_gitapex_check_bash_safety_properties.py | 82 ++++++++ 3 files changed, 277 insertions(+), 16 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 98f4831b..d08232d1 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -982,10 +982,46 @@ def _strip_comments(command: str) -> str: to appear in the comment text) into `checkout_restore_paths` as a phantom candidate, and denying an entirely safe checkout with a misleading message naming a file the command never referenced. Only - an over-denial (never a missed real discard), but a confusing one.""" + an over-denial (never a missed real discard), but a confusing one. + + A double-quoted string's own content delegates to `_consume_double_ + quoted_content` rather than being handled inline -- CRITICAL, full- + classifier-bypass bug found by independent adversarial review (round + 7, issue #1375): the PRIOR inline double-quote handling treated + everything inside an open double quote as opaque literal text with + NO comment recognition at all, correct for genuine literal content + (`"a#b"` really is one literal word in real bash) but WRONG for a + `$(...)` embedded inside that double-quoted string -- real bash + recursively re-enters full, ordinary command grammar for a + substitution's own content regardless of what quote encloses the + `$(` that opened it, so a `#` inside it DOES start a real comment + (confirmed live: a `)` inside a `#`-comment inside `"$(...)"` does + NOT end the substitution). Left unstripped, that comment's own + embedded `)` survived into shlex's dequoted token, where + `_find_fused_command_substitution`'s own paren-depth counter (see + that function's own docstring) -- comment- and quote-blind by + design -- mistook it for the substitution's REAL closing paren, + silently truncating everything after that point, INCLUDING a + genuine, undisguised `git checkout` on the next physical line, from + ALL classification, not merely this module's own checkout/restore + rule. Live-verified real, silent data loss: `x="$(echo hi #comment + with paren ) here` + a real newline + `git checkout -- dirty.py)"` + ran the embedded checkout for real and discarded an uncommitted + change, while `classify()` reported `deny=False` with an EMPTY + `checkout_restore_paths` -- a confident, wrong "nothing to see here" + instead of an honest non-goal, precisely the failure class this + whole module exists to avoid. The analogous decoy built from a + literal `)` inside a nested QUOTED span (rather than a comment) does + NOT need this fix and was checked live: any balanced quoted span + containing a literal `)` necessarily leaves an ODD, unbalanced quote + count in text naively truncated partway through it, which already + trips `tokenize()`'s own `TokenizeError` fail-closed path -- only a + comment can hide an unbalanced `)` without requiring an unbalanced + quote in the truncated prefix, which is why this fix is scoped to + comment-handling specifically rather than a general rewrite of the + paren-depth counter itself.""" out: list[str] = [] in_single_quote = False - in_double_quote = False at_boundary = True i = 0 n = len(command) @@ -998,20 +1034,12 @@ def _strip_comments(command: str) -> str: at_boundary = False i += 1 continue - if in_double_quote: - if char == "\\" and i + 1 < n: - nxt = command[i + 1] - out.append(char) - out.append(nxt) - i += 2 - if nxt != "\n": - at_boundary = False - continue + if char == '"': out.append(char) - if char == '"': - in_double_quote = False - at_boundary = False i += 1 + inner, i = _consume_double_quoted_content(command, i) + out.append(inner) + at_boundary = False continue if char == "\\" and i + 1 < n: nxt = command[i + 1] @@ -1027,10 +1055,134 @@ def _strip_comments(command: str) -> str: i += 1 at_boundary = False continue + if char == "#" and at_boundary: + end = command.find("\n", i) + i = n if end == -1 else end + continue + out.append(char) + at_boundary = char in _COMMENT_BOUNDARY_CHARS + i += 1 + return "".join(out) + + +def _consume_double_quoted_content(command: str, i: int) -> tuple[str, int]: + """Process the content of a double-quoted string starting at + COMMAND[i] (the character right after the opening `"`, already + appended by the caller): everything is literal EXCEPT a nested + `$(...)`, which re-enters ordinary, comment-aware command parsing + via `_consume_command_substitution_content` -- see that function's + own docstring, and `_strip_comments`'s own round-7 addendum, for the + live-verified bypass this closes. Returns (the content up to and + including its own matching `"`, or the remainder of COMMAND if + unterminated -- an unbalanced double quote is `tokenize()`'s own + concern to fail closed on via `TokenizeError`, not this function's, + which only strips comments and never itself validates quote + balance -- the index one past that point).""" + out: list[str] = [] + n = len(command) + while i < n: + char = command[i] if char == '"': - in_double_quote = True out.append(char) i += 1 + return "".join(out), i + if char == "\\" and i + 1 < n: + out.append(char) + out.append(command[i + 1]) + i += 2 + continue + if char == "$" and i + 1 < n and command[i + 1] == "(": + out.append("$(") + i += 2 + inner, i = _consume_command_substitution_content(command, i) + out.append(inner) + continue + out.append(char) + i += 1 + return "".join(out), i + + +def _consume_command_substitution_content(command: str, i: int) -> tuple[str, int]: + """Process the content of a `$(...)` starting at COMMAND[i] (the + character right after the opening `$(`, already appended by the + caller), mirroring real bash's own re-entrant grammar: a command + substitution's own content is parsed as ordinary, top-level shell + text regardless of what quote (if any) encloses the `$(` that opened + it -- comments are live again, and a nested `'`/`"`/`$(` inside gets + its own, independent handling (a nested `"..."` delegates back to + `_consume_double_quoted_content`, which can itself contain a FURTHER + nested `$(...)`, exactly mirroring bash's own mutual recursion + between quote parsing and command parsing). Tracks its own raw, + unquoted paren DEPTH (starting at 1, for the substitution this call + itself is inside) to find its own matching closing `)` -- a nested + unquoted `(`/`)` (a subshell, or arithmetic-looking text this module + does not otherwise interpret) increments/decrements it exactly like + `_find_fused_command_substitution`'s own counter does, but unlike + that counter, a `(`/`)` sitting inside a quote or a stripped comment + here is correctly never counted at all, since this function consumes + those spans as opaque units before ever inspecting their content for + a bare paren. Returns (the content up to and including its own + matching `)`, with every comment inside it deleted, the index one + past that `)`) -- or, if COMMAND ends before depth returns to 0, the + remainder of COMMAND with whatever comments were found still + stripped (an unbalanced `$(...)` is `tokenize()`'s own concern to + fail closed on via `TokenizeError`, not this function's). + + See `_strip_comments`'s own round-7 docstring addendum (issue #1375) + for the live-verified, real-data-loss bypass this function exists to + close, and for why the analogous decoy built from a quoted (rather + than commented) literal `)` needs no fix here.""" + out: list[str] = [] + at_boundary = True + depth = 1 + n = len(command) + while i < n: + char = command[i] + if char == "'": + out.append(char) + i += 1 + while i < n and command[i] != "'": + out.append(command[i]) + i += 1 + if i < n: + out.append(command[i]) + i += 1 + at_boundary = False + continue + if char == '"': + out.append(char) + i += 1 + inner, i = _consume_double_quoted_content(command, i) + out.append(inner) + at_boundary = False + continue + if char == "\\" and i + 1 < n: + nxt = command[i + 1] + out.append(char) + out.append(nxt) + i += 2 + if nxt != "\n": + at_boundary = False + continue + if char == "$" and i + 1 < n and command[i + 1] == "(": + out.append("$(") + i += 2 + inner, i = _consume_command_substitution_content(command, i) + out.append(inner) + at_boundary = False + continue + if char == "(": + depth += 1 + out.append(char) + at_boundary = False + i += 1 + continue + if char == ")": + depth -= 1 + out.append(char) + i += 1 + if depth == 0: + return "".join(out), i at_boundary = False continue if char == "#" and at_boundary: @@ -1040,7 +1192,7 @@ def _strip_comments(command: str) -> str: out.append(char) at_boundary = char in _COMMENT_BOUNDARY_CHARS i += 1 - return "".join(out) + return "".join(out), i def _strip_line_continuations(command: str) -> str: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 6a71c9f0..76466807 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1439,6 +1439,33 @@ def test_checkout_allowed_when_a_comment_after_a_line_continuation_names_an_unre assert result.stderr == "" +def test_checkout_denied_for_a_real_checkout_hidden_behind_a_commented_paren_in_a_substitution( + tmp_path: Path, +) -> None: + """CRITICAL, full-classifier-bypass regression pin (round-7 + independent review, issue #1375). A `$(...)` embedded inside an + outer double-quoted string re-enters ordinary, comment-aware command + parsing in real bash -- a `)` inside a `#`-comment inside it does + NOT end the substitution. The classifier used to not know this, + leaving the comment (and its embedded `)`) unstripped; that stray + `)` then made the command-substitution paren counter mistake it for + the real closing paren, silently dropping everything after it -- + including the real `git checkout` on the next line -- from all + classification. Live-verified before the fix: the embedded checkout + ran for real and discarded an uncommitted change while this wrapper + allowed the command outright.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py", content="ORIGINAL CONTENT\n") + file_path.write_text("UNCOMMITTED LOCAL EDIT\n") + command = 'x="$(echo hi #comment with paren ) here\ngit checkout -- dirty.py)"' + result = run(command, payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert payload["hookSpecificOutput"]["permissionDecision"] == "deny" + assert "dirty.py" in payload["systemMessage"] + assert file_path.read_text() == "UNCOMMITTED LOCAL EDIT\n" + + def test_checkout_denied_from_a_subdirectory_when_target_has_uncommitted_changes(tmp_path: Path) -> None: """The near-miss's own exact shape (issue #1375, issue #1128 repair 4): replayed from a SUBDIRECTORY of the repo, not just the repo root -- diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 5f5276b0..03b0c418 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3308,6 +3308,88 @@ def test_classify_no_longer_leaks_a_comment_past_a_line_continuation() -> None: assert verdict.checkout_restore_paths == ("clean.py",) +def test_strip_comments_strips_a_comment_nested_inside_a_double_quoted_substitution() -> None: + """CRITICAL, full-classifier-bypass regression pin (round-7 + independent review, issue #1375). Real bash recursively re-enters + ordinary, comment-aware command parsing for a `$(...)` embedded + inside an outer double-quoted string -- a `)` inside a `#`-comment + inside such a substitution does NOT end the substitution. The old + inline double-quote handling here did not know this, leaving the + comment (and its embedded `)`) unstripped; that stray `)` then made + `_find_fused_command_substitution`'s own naive paren counter mistake + it for the substitution's real closing paren, silently truncating + everything after it -- including a real `git checkout` on the next + line -- from all classification. Live-verified this let a genuine, + dirty-file checkout run for real while `classify()` reported + `deny=False` with an empty `checkout_restore_paths`.""" + result = checker._strip_comments('x="$(echo hi #comment with paren ) here\ngit checkout -- dirty.py)"') + assert result == 'x="$(echo hi \ngit checkout -- dirty.py)"' + + +def test_classify_no_longer_loses_a_checkout_behind_a_commented_paren_in_a_substitution() -> None: + """End-to-end regression pin for the round-7 finding at the + `classify()` level: the real `git checkout -- dirty.py` embedded + past the decoy comment must be found, not silently dropped.""" + verdict = checker.classify('x="$(echo hi #comment with paren ) here\ngit checkout -- dirty.py)"') + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + +def test_strip_comments_strips_a_comment_inside_a_substitution_nested_two_levels_deep() -> None: + """No shortcut taken for nesting depth: a comment inside a `$(...)` + that is itself nested inside ANOTHER `$(...)` inside the outer + double-quoted string must also be recognized and stripped, mirroring + bash's own arbitrarily-recursive re-entrant grammar.""" + result = checker._strip_comments('x="$(echo $(echo hi #comment\n) tail)"') + assert result == 'x="$(echo $(echo hi \n) tail)"' + + +def test_strip_comments_still_treats_a_literal_hash_inside_a_substitution_string_as_literal() -> None: + """No over-stripping regression: a `#` inside a QUOTED span within + the substitution's own content is still ordinary literal text, not a + comment-starter -- matching the same rule this function already + enforces at the top level.""" + result = checker._strip_comments("x=\"$(echo 'a#b' tail)\"") + assert result == "x=\"$(echo 'a#b' tail)\"" + + +@pytest.mark.parametrize( + "command", + [ + 'x="$(echo \'abc)"', + 'x="$(echo "inner" tail)"', + 'x="$(echo a\\\\x)"', + 'x="$(echo (nested) tail)"', + 'x="$(echo abc', + ], + ids=[ + "unterminated-single-quote-inside-substitution", + "nested-double-quote-inside-substitution", + "non-newline-escape-inside-substitution", + "nested-unquoted-parens-inside-substitution", + "unterminated-substitution", + ], +) +def test_strip_comments_is_a_no_op_without_a_comment_inside_a_substitution(command: str) -> None: + """No crash and no unintended stripping on every other shape + `_consume_command_substitution_content` must walk through correctly + to find (or fail to find, for the unterminated case) its own + matching close-paren -- none of these contain a `#`, so the result + must be byte-for-byte identical to the input; a real unbalanced + quote/substitution is `tokenize()`'s own concern to fail closed on, + not this function's, which never itself validates balance.""" + assert checker._strip_comments(command) == command + + +def test_strip_comments_preserves_a_line_continuation_inside_a_substitution() -> None: + """The round-6 boundary-preserving fix applies inside a substitution + too, not only at the top level: a genuine `\\` there must + not clear AT_BOUNDARY, so a `#`-comment right after it is still + correctly recognized and stripped.""" + result = checker._strip_comments('x="$(echo a \\\nb #c\nd)"') + assert result == 'x="$(echo a \\\nb \nd)"' + + @pytest.mark.parametrize("flag", ["-b", "-B", "--orphan"]) def test_git_checkout_paths_folds_branch_creation_flags_into_the_non_goal(flag: str) -> None: """CRITICAL regression pin (round-4 independent review, issue #1375). From 3c710d596730be04c2aca841ec2382166b78b30d Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 13:13:24 +0000 Subject: [PATCH 13/46] docs(hooks): disclose a critical, whole-module shlex nested-quote bypass A fresh, independent adversarial review of this PR's current head (round 8), stress-testing the round-7 comment/substitution-nesting fix, found a distinct, more fundamental bug: Python's shlex module (which tokenize() relies on) tracks double-quote state as one flat, whole-command toggle, with no concept of bash's own recursive quote-context reset inside a $(...) command substitution. A double-quoted span nested inside a $(...) that is itself nested inside an outer double-quoted string desynchronizes shlex's own quote parity from real bash's actual parse, while keeping the total double-quote count even across the whole command -- so tokenize()'s own TokenizeError fail-closed path never fires, unlike the structurally-safe, always-unbalanced quote-decoy shape the round-7 fix's own docstring already documents. Live-verified real, silent data loss: x="$(echo "y)" && git checkout -- dirty.py)" genuinely discards a dirty tracked file when actually executed, while classify() reports deny=False with "git"/"checkout" never appearing as their own separate tokens at all -- fused into what shlex mis-reads as inert quoted content. This is a property of shlex itself, not any rule built on top of it: every existing rule (checkout/restore, git push, pip install, gh api) shares this exposure equally, and it reproduces on commits predating this PR's own round-7/round-8 fixes -- confirmed not introduced by issue #1375's own checkout/restore feature. A genuine fix needs a command-substitution-aware recursive tokenizer replacing shlex's own single-pass, non-recursive quote state machine, not a narrow patch -- disproportionate scope for issue #1375's own checkout/restore feature. Tracked as its own dedicated issue, https://github.com/tvna/gitapex/issues/1404, rather than fixed here. Disclosed in the module's own header docstring and pinned as a tested, current-behavior regression in KNOWN_BYPASS_COMMANDS (shlex-nested-double-quote-inside-command-substitution-full-bypass), matching this module's own established convention for a disclosed, accepted residual rather than leaving it silently uncovered. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 26 +++++++++++++++++++++ hooks/test_gitapex_check_bash_safety.py | 31 +++++++++++++++++++++++++ 2 files changed, 57 insertions(+) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index d08232d1..a5532ed0 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -79,6 +79,32 @@ pinned as `graphql-mutation-keyword-variable-concatenation` in hooks/test_gitapex_check_bash_safety.py's own `KNOWN_BYPASS_COMMANDS`. +CRITICAL, disclosed, whole-module limitation, NOT specific to any one rule +(found live by Step 8 independent review, round 8 of issue #1375's own +checkout/restore feature review, while stress-testing an unrelated, +narrower fix): `tokenize()`'s own reliance on the standard library's +`shlex` tracks double-quote state as one flat, whole-command toggle, with +no concept of bash's own recursive quote-context reset inside a `$(...)` +command substitution. A double-quoted span nested inside a `$(...)` that +is itself nested inside an outer double-quoted string desynchronizes +`shlex`'s own quote parity from real bash's actual parse while keeping +the TOTAL double-quote count even across the whole command -- so +`tokenize()`'s own `TokenizeError` fail-closed path never fires, unlike +the structurally-safe, always-unbalanced quote-decoy shape documented +elsewhere in this module. Live-verified real, silent data loss: `x="$(echo +"y)" && git checkout -- dirty.py)"` genuinely discards a dirty tracked +file when actually executed, while `classify()` reports `deny=False` with +"git"/"checkout" never appearing as their own separate tokens at all -- +fused into what `shlex` mis-reads as inert quoted content. This is a +property of `shlex` itself, not any rule built on top of it (checkout/ +restore, git push, pip install, gh api all share this exposure equally), +and predates issue #1375. Deliberately NOT attempted here -- tracked as +its own dedicated issue, https://github.com/tvna/gitapex/issues/1404, +since a genuine fix needs a command-substitution-aware recursive +tokenizer, not a narrow patch; pinned as `shlex-nested-double-quote- +inside-command-substitution-full-bypass` in +hooks/test_gitapex_check_bash_safety.py's own `KNOWN_BYPASS_COMMANDS`. + Closed by fifth-round Step 8 independent review: `_gh_api_method_dynamic_ value`/`_gh_api_field_dynamic_hit` (and the earlier literal-token scans) only ever recognized a dynamic VALUE fused onto a literal `-X`/`--method`/ diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 76466807..45a6aed8 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -417,6 +417,37 @@ def assert_allowed(command: str) -> None: "git checkout -f -b newbranch other", "checkout-branch-creation-flag-non-goal", ), + ( + # CRITICAL, WHOLE-MODULE bypass -- NOT specific to checkout/restore + # or to this file's own KNOWN_BYPASS_COMMANDS convention's usual + # narrow-decoy shape. Found live by independent adversarial review + # (round 8, issue #1375), tracked as its own dedicated issue rather + # than fixed here: https://github.com/tvna/gitapex/issues/1404 -- + # deliberately out of issue #1375's own scope, since the root cause + # predates it, is architectural (Python's `shlex` tracks + # double-quote state as one flat, whole-command toggle with no + # concept of bash's own recursive quote-context reset inside a + # `$(...)` command substitution), and is shared by EVERY rule in + # this file, not just checkout/restore. A double-quoted span + # nested inside a `$(...)` that is itself nested inside an outer + # double-quoted string desynchronizes `shlex`'s quote parity from + # real bash's own parse while keeping the TOTAL quote-character + # count even, so `tokenize()`'s own `TokenizeError` fail-closed + # path never fires (unlike the structurally-safe, always- + # unbalanced quote-decoy case the round-7 fix's own docstring + # already documents). Live-verified real, silent data loss: this + # exact command genuinely discards a dirty tracked file named + # `dirty.py` when actually executed, while `classify()` reports + # `deny=False` with an EMPTY `checkout_restore_paths` -- "git" + # and "checkout" never appear as their own separate tokens at all, + # fused into an inert-looking quoted blob by `shlex`'s own + # mis-toggled state. See issue #1404 for the full write-up, + # live-verification detail, and why a genuine fix needs a + # command-substitution-aware recursive tokenizer rather than a + # narrow patch. + 'x="$(echo "y)" && git checkout -- dirty.py)"', + "shlex-nested-double-quote-inside-command-substitution-full-bypass", + ), ] From e4138ce8f88616735df084bb055307376307552b Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 13:28:24 +0000 Subject: [PATCH 14/46] fix(hooks): recognize pushd/popd alongside cd as cwd-relocating A fresh, independent adversarial review of this PR's current head (round 9), verifying the round-8 disclosure commit and sweeping once more for anything else, found that _rule_git_checkout_restore only recognized a literal cd token as making the wrapper's own fixed .cwd reference point unsound for a later checkout/restore segment. pushd relocates the shell's own working directory exactly the same way cd does, and was not recognized at all. Live-verified: with a target file dirty relative to a pushd-ed subdirectory but absent at the PreToolUse payload's own .cwd, `pushd sub && git checkout -- dirty.py` resolved checkout_restore_paths to a confident, wrong claim (the literal filename, checked against the wrong tree), the wrapper's live check found nothing at that wrong path and allowed the command, and the real command silently discarded the uncommitted change when actually executed -- the same "confident wrong claim" failure class several earlier rounds already closed for other flag/token gaps. Fixed by widening the existing saw_cd check to a new _CWD_RELOCATING_COMMANDS set covering cd, pushd, and popd (popd also relocates the shell's cwd, to whatever the directory stack's own prior entry was, which this classifier has no way to know either). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 41 ++++++++++++++----- hooks/test_gitapex_check_bash_safety.py | 21 ++++++++++ ...st_gitapex_check_bash_safety_properties.py | 26 ++++++++++++ 3 files changed, 77 insertions(+), 11 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index a5532ed0..1f2c3bfe 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3594,6 +3594,9 @@ def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str] return None, [], False +_CWD_RELOCATING_COMMANDS = {"cd", "pushd", "popd"} + + def _rule_git_checkout_restore( segments: list[list[str]], raw_assigned: dict[str, str] ) -> tuple[str | None, tuple[str, ...]]: @@ -3602,27 +3605,43 @@ def _rule_git_checkout_restore( classifier cannot soundly determine which working tree is at risk: a `-C`/`--git-dir`/`--work-tree` global flag on the checkout/restore segment itself, a `GIT_DIR=`/`GIT_WORK_TREE=`/`GIT_INDEX_FILE=` - assignment anywhere in the command, or a literal `cd` in an earlier - segment of the same command. hooks/check-bash-safety.sh's own new - wrapper step always checks a path against `.cwd` from the PreToolUse - payload (issue #1375's own Fact 5, the cwd-mismatch finding) -- any of - these makes that single, fixed `.cwd` reference point unsound for this - particular invocation, so this denies here (I/O-free -- a token-shape - fact, not a live check) rather than letting the wrapper check the - wrong tree.""" + assignment anywhere in the command, or a literal `cd`/`pushd`/`popd` + in an earlier segment of the same command. hooks/check-bash-safety.sh's + own new wrapper step always checks a path against `.cwd` from the + PreToolUse payload (issue #1375's own Fact 5, the cwd-mismatch + finding) -- any of these makes that single, fixed `.cwd` reference + point unsound for this particular invocation, so this denies here + (I/O-free -- a token-shape fact, not a live check) rather than letting + the wrapper check the wrong tree. + + `pushd`/`popd` join `cd` in `_CWD_RELOCATING_COMMANDS` -- CRITICAL bug + found by independent adversarial review (round 9, issue #1375) and + independently reproduced live: only a literal `cd` token was + recognized here, but `pushd ` relocates the shell's own working + directory exactly like `cd` does (confirmed live: `pushd sub && + git checkout -- dirty.py`, with `dirty.py` dirty relative to `sub` + but absent at the PreToolUse payload's own `.cwd`, resolved + `checkout_restore_paths` to `('dirty.py',)` -- a CONFIDENT, WRONG + claim, since the wrapper's live `git diff` check against that + filename at the wrong `.cwd` found no such path and reported clean -- + and the real command silently discarded the uncommitted change when + actually executed). `popd` joins for the same reason: it also + relocates the shell's cwd, to whatever the directory stack's own + prior entry was, which this classifier has no way to know either.""" saw_cd = False all_paths: list[str] = [] for seg in segments: subcommand, tokens_after, saw_tree_relocation = _find_git_checkout_restore(seg, raw_assigned) if subcommand is None: - if any(not _is_dynamic(t) and t == "cd" for t in seg): + if any(not _is_dynamic(t) and t in _CWD_RELOCATING_COMMANDS for t in seg): saw_cd = True continue if saw_tree_relocation or saw_cd or any(name in raw_assigned for name in _GIT_TREE_ENV_VARS): return ( f"a 'git {subcommand}' command carries a -C/--git-dir/--work-tree flag, a GIT_DIR=/" - "GIT_WORK_TREE=/GIT_INDEX_FILE= assignment, or an earlier 'cd' in the same command -- this " - "classifier cannot soundly determine which working tree is at risk, so this is denied outright", + "GIT_WORK_TREE=/GIT_INDEX_FILE= assignment, or an earlier 'cd'/'pushd'/'popd' in the same " + "command -- this classifier cannot soundly determine which working tree is at risk, so this " + "is denied outright", (), ) if subcommand == "checkout": diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 45a6aed8..00106e4b 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1647,6 +1647,27 @@ def test_checkout_with_tree_relocation_flag_denied_end_to_end(tmp_path: Path) -> assert "working tree is at risk" in payload["systemMessage"] +def test_checkout_denied_when_an_earlier_pushd_relocates_the_working_tree(tmp_path: Path) -> None: + """CRITICAL regression pin (round-9 independent review, issue #1375). + `pushd` relocates the shell's own working directory exactly like `cd` + does, but only `cd` was recognized here. Live-verified before the + fix: with a target file dirty relative to a subdirectory but absent + at the repo root (the PreToolUse payload's own `.cwd`), the wrapper + allowed `pushd sub && git checkout -- dirty.py` outright (the + classifier's own claimed `checkout_restore_paths` checked the wrong + tree and found nothing) and the real command silently discarded the + uncommitted change. The deny here is classifier-level (a token-shape + fact, no live git call), so no such file needs to actually exist for + this regression pin -- the whole point is that it never gets far + enough to check.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("pushd sub && git checkout -- dirty.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 03b0c418..25fac618 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3510,6 +3510,32 @@ def test_rule_git_checkout_restore_allows_cd_after_the_checkout_segment() -> Non assert resolved == ("f.py",) +@pytest.mark.parametrize("relocator", ["pushd", "popd"]) +def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_pushd_or_popd(relocator: str) -> None: + """CRITICAL regression pin (round-9 independent review, issue #1375). + `pushd`/`popd` relocate the shell's own working directory exactly + like `cd` does, but only a literal `cd` token was recognized here -- + live-verified this let `pushd sub && git checkout -- dirty.py` + (dirty.py dirty relative to `sub`, absent at the PreToolUse payload's + own `.cwd`) resolve to a CONFIDENT, WRONG `checkout_restore_paths` + claim that the wrapper's live check then found clean at the wrong + `.cwd`, silently allowing a real, uncommitted-change discard.""" + segments = [[relocator, "/tmp"], ["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {}) + assert reason is not None + assert resolved == () + + +def test_classify_denies_a_checkout_hidden_behind_pushd() -> None: + """End-to-end regression pin for the round-9 finding at the + `classify()` level: the previously wrong, confident + `checkout_restore_paths=('dirty.py',)` claim must become an honest + outright deny instead.""" + verdict = checker.classify("pushd sub && git checkout -- dirty.py") + assert verdict.deny is True + assert "pushd" in verdict.reason + + # --- End-to-end classify() coverage, pinning every explicit safe/deny case # issue #1375's own Acceptance Criteria Map and "Explicit safe cases" # section name by hand. From 7fe95a26cebcda94fed89e3ba73fd77c0fbad6df Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 13:49:48 +0000 Subject: [PATCH 15/46] fix(hooks): recognize a dynamic cd/pushd/popd as cwd-relocating too A fresh, independent adversarial review of this PR's current head (round 10) found that round 9's own fix only ever recognized a literal `cd`/`pushd`/`popd` token written out directly in a segment. A dynamic command word that resolves to one of those at real bash runtime was not recognized at all. Live-verified: `X=cd; $X sub; git checkout -- file.py` (and the same shape with `X=pushd`), with the target file dirty relative to `sub` but absent at the PreToolUse payload's own `.cwd`, resolved checkout_restore_paths to a confident, wrong claim (the literal filename, checked against the wrong tree) -- the same "confident wrong claim" failure class every earlier round in this PR has closed for a different gap. The wrapper's live check then found nothing at that wrong path and allowed the command, and the real command silently discarded the uncommitted change when actually executed. Fixed by additionally flagging a segment whose (already assignment-stripped) command word is dynamic and does not unambiguously vanish at real bash runtime, reusing the same _token_is_all_unassigned_refs/_token_is_a_vanishing_default_or_alt_clause primitives _find_git_checkout_restore already uses for the analogous git/subcommand-position question. A token that genuinely vanishes is deliberately left unflagged here, since bash would then run whatever follows as the real command word, and the existing literal scan already covers a literal cd/pushd/popd sitting after such a decoy. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 33 ++++++++++++++- hooks/test_gitapex_check_bash_safety.py | 18 +++++++++ ...st_gitapex_check_bash_safety_properties.py | 40 +++++++++++++++++++ 3 files changed, 89 insertions(+), 2 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 1f2c3bfe..79fc620a 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3627,13 +3627,42 @@ def _rule_git_checkout_restore( and the real command silently discarded the uncommitted change when actually executed). `popd` joins for the same reason: it also relocates the shell's cwd, to whatever the directory stack's own - prior entry was, which this classifier has no way to know either.""" + prior entry was, which this classifier has no way to know either. + + A DYNAMIC command word at `seg[0]` (after `_classify_tokens`'s own + uniform `_strip_leading_assignments`, so `seg[0]` is always the real + command word here) that does not unambiguously vanish is ALSO treated + as a possible relocator -- CRITICAL bug found by independent + adversarial review (round 10, issue #1375) and independently + reproduced live: the literal-token scan above only ever recognized + `cd`/`pushd`/`popd` written out directly, so `X=cd; $X sub; git + checkout -- file.py` (dirty relative to `sub`, absent at the + PreToolUse payload's own `.cwd`) resolved to the same CONFIDENT, + WRONG `checkout_restore_paths` claim round 9's fix closed for the + literal case, and the real command silently discarded the + uncommitted change when actually executed; same result for `X=pushd`. + A token that unambiguously vanishes (`_token_is_all_unassigned_refs`/ + `_token_is_a_vanishing_default_or_alt_clause`, the same primitives + `_find_git_checkout_restore` already uses for the identical git/ + subcommand-position question) is NOT flagged here, since real bash + then runs whatever token follows as the actual command word instead + -- and the existing literal scan above, which checks every token in + the segment regardless of position, already covers a literal `cd`/ + `pushd`/`popd` sitting after such a decoy without this addition + needing its own skip-past loop.""" saw_cd = False all_paths: list[str] = [] for seg in segments: subcommand, tokens_after, saw_tree_relocation = _find_git_checkout_restore(seg, raw_assigned) if subcommand is None: - if any(not _is_dynamic(t) and t in _CWD_RELOCATING_COMMANDS for t in seg): + if any(not _is_dynamic(t) and t in _CWD_RELOCATING_COMMANDS for t in seg) or ( + seg + and _is_dynamic(seg[0]) + and not ( + _token_is_all_unassigned_refs(seg[0], raw_assigned) + or _token_is_a_vanishing_default_or_alt_clause(seg[0], raw_assigned) + ) + ): saw_cd = True continue if saw_tree_relocation or saw_cd or any(name in raw_assigned for name in _GIT_TREE_ENV_VARS): diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 00106e4b..34a333fb 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1668,6 +1668,24 @@ def test_checkout_denied_when_an_earlier_pushd_relocates_the_working_tree(tmp_pa assert "working tree is at risk" in payload["systemMessage"] +def test_checkout_denied_when_an_earlier_dynamic_word_resolves_to_cd(tmp_path: Path) -> None: + """CRITICAL regression pin (round-10 independent review, issue #1375). + Round 9's fix only recognized a literal `cd`/`pushd`/`popd` token -- + a dynamic command word that resolves to one of those at real bash + runtime (`X=cd; $X ...`) was not recognized at all. Live-verified + before this fix: the classifier's own claimed `checkout_restore_paths` + would have checked the wrong tree, letting the wrapper's live `git + diff` check silently allow a real, uncommitted-change discard -- + classifier-level deny (no live git call), so no such file needs to + actually exist for this regression pin.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("X=cd; $X sub; git checkout -- dirty.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 25fac618..39e86929 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3536,6 +3536,46 @@ def test_classify_denies_a_checkout_hidden_behind_pushd() -> None: assert "pushd" in verdict.reason +def test_rule_git_checkout_restore_denies_when_an_earlier_segment_starts_with_a_dynamic_non_vanishing_word() -> None: + """CRITICAL regression pin (round-10 independent review, issue + #1375). Round 9's literal-token scan only ever recognized + `cd`/`pushd`/`popd` written out directly -- a dynamic command word + (e.g. `$X` with `X=cd`) that resolves to one of those at real bash + runtime was not recognized at all, live-verified to let `X=cd; $X + sub; git checkout -- file.py` resolve to a CONFIDENT, WRONG + `checkout_restore_paths` claim the same way round 9's own fix closed + for the literal case.""" + segments = [["$X", "sub"], ["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}) + assert reason is not None + assert resolved == () + + +def test_rule_git_checkout_restore_allows_a_genuinely_vanishing_dynamic_word() -> None: + """No false positive: a dynamic `seg[0]` that unambiguously vanishes + (word-splits to nothing at real bash runtime, e.g. an unset + parameter with no default) is NOT flagged as a possible relocator -- + real bash would run whatever token follows as the actual command + word instead, and that token is scanned on its own merits.""" + segments = [["${NEVERSET}", "sub"], ["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {}) + assert reason is None + assert resolved == ("f.py",) + + +def test_classify_denies_a_checkout_hidden_behind_a_dynamic_cd() -> None: + """End-to-end regression pin for the round-10 finding at the + `classify()` level: a variable holding `cd` (or `pushd`) must deny + the same way a literal `cd`/`pushd` does.""" + for cmd in ( + "X=cd; $X sub; git checkout -- file.py", + "X=pushd; $X sub; git checkout -- file.py", + ): + verdict = checker.classify(cmd) + assert verdict.deny is True, cmd + assert verdict.checkout_restore_paths == () + + # --- End-to-end classify() coverage, pinning every explicit safe/deny case # issue #1375's own Acceptance Criteria Map and "Explicit safe cases" # section name by hand. From 9f6393e02f6e2e9c62f89f719ff4616d4db4803b Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 14:09:09 +0000 Subject: [PATCH 16/46] fix(hooks): narrow dynamic-cd detection to actually resolvable cd/pushd/popd A fresh, independent adversarial review of this PR's current head (round 11) found that round 10's own fix -- flagging every dynamic, non-vanishing command word as a possible cwd-relocator -- was itself over-broad. It denied ordinary, unrelated commands with no cwd- relocation risk at all, purely because their first word happened to be dynamic. Live-verified: `EDITOR=vim; $EDITOR sub; git checkout -- f.py` (a completely safe `$EDITOR`/`$TOOL` dispatch idiom followed by an unrelated, clean checkout) was denied outright with the same "cannot soundly determine which working tree is at risk" reason meant for a genuine `X=cd`-style bypass. This is the same over-broad "deny every dynamic word" policy this module's own header docstring already measured at a 28% false-positive rate and rejected everywhere else in the file; round 10 had reintroduced it in this one spot. Fixed by adding `_dynamic_word_may_resolve_to_a_cwd_relocator`, which actually resolves the dynamic word's candidate value(s) via the existing `_substitute_var_refs_candidates` primitive (reused with the case-preserved raw-value map for both of its parameters, since `cd`/`pushd`/`popd` are real, case-sensitive bash command names, unlike the case-insensitive write-method literals every other caller of that primitive compares) and flags only when a candidate could genuinely be `cd`/`pushd`/`popd`, or resolution is itself ambiguous or unresolvable. A token whose dynamism cannot be decomposed into `$NAME`-shaped references at all (e.g. a folded command-substitution placeholder) still fails closed, preserving round 10's original blanket-flag behavior for that shape exactly -- this fix only ever narrows what round 10 already flagged, never widens it, so round 10's own `X=cd`/`X=pushd` true positives are unaffected. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 105 ++++++++++++++---- hooks/test_gitapex_check_bash_safety.py | 16 +++ ...st_gitapex_check_bash_safety_properties.py | 72 ++++++++++++ 3 files changed, 174 insertions(+), 19 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 79fc620a..c1d70520 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3597,6 +3597,58 @@ def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str] _CWD_RELOCATING_COMMANDS = {"cd", "pushd", "popd"} +def _dynamic_word_may_resolve_to_a_cwd_relocator(token: str, name_to_raw_value: dict[str, str]) -> bool: + """Whether a DYNAMIC command word (already confirmed non-vanishing by + the caller) could plausibly resolve, at real bash runtime, to a + literal `cd`/`pushd`/`popd` -- narrower than round 10's own original + fix, which flagged EVERY non-vanishing dynamic `seg[0]` regardless of + what it could actually resolve to. + + CRITICAL false-positive bug found by independent adversarial review + (round 11, issue #1375) and independently reproduced live: round 10's + blanket flag denied `EDITOR=vim; $EDITOR sub; git checkout -- f.py` + outright -- a completely safe, ordinary command (an `$EDITOR`/`$TOOL`/ + positional-parameter dispatch idiom, followed by an unrelated, clean + checkout) -- purely because `$EDITOR` is dynamic and does not vanish, + with no attempt to check what it could actually resolve to. This is + the same over-broad "deny every dynamic word" policy this module's + own opening docstring already measured at a 28% false-positive rate + and rejected everywhere else; round 10 had reintroduced it in this one + narrow spot. + + Resolves TOKEN via `_substitute_var_refs_candidates`, reusing it with + NAME_TO_RAW_VALUE passed as BOTH of that function's parameters (rather + than the module's usual lowercased `name_to_value`) so every candidate + stays case-PRESERVED -- `cd`/`pushd`/`popd` are real bash command + names, case-SENSITIVE unlike the write-method literals every other + caller of that primitive compares case-insensitively; lowercasing here + would make an assignment like `X=CD` (which real bash would try to run + as literal, non-existent command `CD`, not the `cd` builtin) a false + positive of its own. + + Three cases: + - No `$NAME`-shaped reference found in TOKEN at all + (`_VAR_REF_FULL_RE.search` finds nothing) -- TOKEN's dynamism comes + from something this resolution primitive cannot decompose (e.g. a + folded command-substitution placeholder). Fails closed (`True`), + preserving round 10's own blanket-flag behavior for this shape + exactly -- this function only ever NARROWS what round 10 already + flagged, never widens it. + - `_substitute_var_refs_candidates` returns `None` (too many candidate + readings to enumerate) or `[]` (some referenced name has no + assigned-and-in-range reading this classifier can resolve) -- both + genuine ambiguity, not a resolved-safe value. Fails closed (`True`). + - A concrete candidate list -- flags (`True`) only if some candidate + is exactly `cd`/`pushd`/`popd`; otherwise the word demonstrably + resolves to something else, so returns `False`.""" + if _VAR_REF_FULL_RE.search(token) is None: + return True + candidates = _substitute_var_refs_candidates(token, name_to_raw_value, name_to_raw_value) + if candidates is None or not candidates: + return True + return any(candidate in _CWD_RELOCATING_COMMANDS for candidate in candidates) + + def _rule_git_checkout_restore( segments: list[list[str]], raw_assigned: dict[str, str] ) -> tuple[str | None, tuple[str, ...]]: @@ -3631,25 +3683,39 @@ def _rule_git_checkout_restore( A DYNAMIC command word at `seg[0]` (after `_classify_tokens`'s own uniform `_strip_leading_assignments`, so `seg[0]` is always the real - command word here) that does not unambiguously vanish is ALSO treated - as a possible relocator -- CRITICAL bug found by independent - adversarial review (round 10, issue #1375) and independently - reproduced live: the literal-token scan above only ever recognized - `cd`/`pushd`/`popd` written out directly, so `X=cd; $X sub; git - checkout -- file.py` (dirty relative to `sub`, absent at the - PreToolUse payload's own `.cwd`) resolved to the same CONFIDENT, - WRONG `checkout_restore_paths` claim round 9's fix closed for the - literal case, and the real command silently discarded the - uncommitted change when actually executed; same result for `X=pushd`. - A token that unambiguously vanishes (`_token_is_all_unassigned_refs`/ - `_token_is_a_vanishing_default_or_alt_clause`, the same primitives - `_find_git_checkout_restore` already uses for the identical git/ - subcommand-position question) is NOT flagged here, since real bash - then runs whatever token follows as the actual command word instead - -- and the existing literal scan above, which checks every token in - the segment regardless of position, already covers a literal `cd`/ - `pushd`/`popd` sitting after such a decoy without this addition - needing its own skip-past loop.""" + command word here) that does not unambiguously vanish, AND could + plausibly resolve to `cd`/`pushd`/`popd`, is ALSO treated as a + possible relocator -- CRITICAL bug found by independent adversarial + review (round 10, issue #1375) and independently reproduced live: the + literal-token scan above only ever recognized `cd`/`pushd`/`popd` + written out directly, so `X=cd; $X sub; git checkout -- file.py` + (dirty relative to `sub`, absent at the PreToolUse payload's own + `.cwd`) resolved to the same CONFIDENT, WRONG `checkout_restore_paths` + claim round 9's fix closed for the literal case, and the real command + silently discarded the uncommitted change when actually executed; + same result for `X=pushd`. A token that unambiguously vanishes + (`_token_is_all_unassigned_refs`/`_token_is_a_vanishing_default_or_ + alt_clause`, the same primitives `_find_git_checkout_restore` already + uses for the identical git/subcommand-position question) is NOT + flagged here, since real bash then runs whatever token follows as the + actual command word instead -- and the existing literal scan above, + which checks every token in the segment regardless of position, + already covers a literal `cd`/`pushd`/`popd` sitting after such a + decoy without this addition needing its own skip-past loop. + + Round 10's own first version flagged EVERY non-vanishing dynamic + `seg[0]`, with no attempt to check what it could actually resolve to + -- CRITICAL false-positive bug found by independent adversarial review + (round 11, issue #1375) and independently reproduced live: + `EDITOR=vim; $EDITOR sub; git checkout -- f.py`, a completely safe, + ordinary command with an unrelated, clean checkout, was denied + outright purely because `$EDITOR` is dynamic and non-vanishing. + `_dynamic_word_may_resolve_to_a_cwd_relocator` narrows this to + actually resolve the word's candidate value(s) (see its own + docstring) and only flags when a candidate could genuinely be + `cd`/`pushd`/`popd`, or resolution is itself ambiguous/unresolvable -- + never widening beyond what round 10 already flagged, only narrowing + it.""" saw_cd = False all_paths: list[str] = [] for seg in segments: @@ -3662,6 +3728,7 @@ def _rule_git_checkout_restore( _token_is_all_unassigned_refs(seg[0], raw_assigned) or _token_is_a_vanishing_default_or_alt_clause(seg[0], raw_assigned) ) + and _dynamic_word_may_resolve_to_a_cwd_relocator(seg[0], raw_assigned) ): saw_cd = True continue diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 34a333fb..997e37b0 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1686,6 +1686,22 @@ def test_checkout_denied_when_an_earlier_dynamic_word_resolves_to_cd(tmp_path: P assert "working tree is at risk" in payload["systemMessage"] +def test_checkout_allowed_when_an_earlier_dynamic_word_resolves_to_something_harmless(tmp_path: Path) -> None: + """CRITICAL false-positive regression pin (round-11 independent + review, issue #1375). Round 10's own first version flagged EVERY + non-vanishing dynamic `seg[0]`, regardless of what it could actually + resolve to -- live-verified before this fix to wrongly deny + `EDITOR=vim; $EDITOR sub; git checkout -- f.py`, a completely safe, + ordinary command (an `$EDITOR`/`$TOOL` dispatch idiom followed by an + unrelated, clean checkout), purely because `$EDITOR` is dynamic and + non-vanishing. The target file here is committed and clean, so a + correct verdict allows the command outright end-to-end.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("EDITOR=vim; $EDITOR sub; git checkout -- f.py", payload_cwd=str(repo_dir)) + assert result.returncode == 0, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 39e86929..4a7799df 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3576,6 +3576,78 @@ def test_classify_denies_a_checkout_hidden_behind_a_dynamic_cd() -> None: assert verdict.checkout_restore_paths == () +def test_rule_git_checkout_restore_allows_a_dynamic_word_resolving_to_something_harmless() -> None: + """CRITICAL false-positive regression pin (round-11 independent + review, issue #1375). Round 10's own first version flagged EVERY + non-vanishing dynamic `seg[0]` regardless of what it could actually + resolve to -- live-verified to wrongly deny `EDITOR=vim; $EDITOR sub; + git checkout -- f.py`, a completely safe, ordinary command (an + `$EDITOR`/`$TOOL` dispatch idiom followed by an unrelated, clean + checkout), purely because `$EDITOR` is dynamic and non-vanishing. + `_dynamic_word_may_resolve_to_a_cwd_relocator` must resolve the + word's actual candidate value and only flag when it could genuinely + be `cd`/`pushd`/`popd`.""" + segments = [["$EDITOR", "sub"], ["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}) + assert reason is None + assert resolved == ("f.py",) + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_VALUES) +def test_dynamic_word_may_resolve_to_a_cwd_relocator_matches_relocator_set(name: str, value: str) -> None: + """Model-based: `_dynamic_word_may_resolve_to_a_cwd_relocator` flags a + resolvable dynamic word if and only if its resolved value is exactly + one of `cd`/`pushd`/`popd` -- case-sensitively, matching real bash's + own case-sensitive command-name lookup.""" + result = checker._dynamic_word_may_resolve_to_a_cwd_relocator(f"${name}", {name: value}) + assert result == (value in checker._CWD_RELOCATING_COMMANDS) + + +def test_dynamic_word_may_resolve_to_a_cwd_relocator_true_for_a_matching_assignment() -> None: + assert checker._dynamic_word_may_resolve_to_a_cwd_relocator("$X", {"X": "cd"}) is True + assert checker._dynamic_word_may_resolve_to_a_cwd_relocator("$X", {"X": "pushd"}) is True + + +def test_dynamic_word_may_resolve_to_a_cwd_relocator_false_for_a_harmless_assignment() -> None: + assert checker._dynamic_word_may_resolve_to_a_cwd_relocator("$EDITOR", {"EDITOR": "vim"}) is False + + +def test_dynamic_word_may_resolve_to_a_cwd_relocator_is_case_sensitive() -> None: + """`cd`/`pushd`/`popd` are real bash command names, case-SENSITIVE -- + an assignment of `CD` (uppercase) must not be treated as resolving to + the `cd` builtin, unlike this module's usual lowercased write-method + comparisons elsewhere.""" + assert checker._dynamic_word_may_resolve_to_a_cwd_relocator("$X", {"X": "CD"}) is False + + +def test_dynamic_word_may_resolve_to_a_cwd_relocator_true_when_unresolvable() -> None: + """A token whose dynamism this classifier cannot decompose into + `$NAME`-shaped references at all (e.g. a folded command-substitution + placeholder) fails closed, preserving round 10's own blanket-flag + behavior for this shape -- this primitive only ever narrows what + round 10 already flagged, never widens it.""" + assert checker._dynamic_word_may_resolve_to_a_cwd_relocator("__CMDSUB_PLACEHOLDER__", {}) is True + + +def test_dynamic_word_may_resolve_to_a_cwd_relocator_true_for_an_unresolvable_reference() -> None: + """A `$NAME`-shaped reference this classifier cannot resolve at all + (NAME never assigned) also fails closed, via + `_substitute_var_refs_candidates`'s own empty-list return -- a + narrower unit-level pin than the vanishing-check short-circuit + `_rule_git_checkout_restore` applies before ever reaching this helper + in that context.""" + assert checker._dynamic_word_may_resolve_to_a_cwd_relocator("$NEVERSET", {}) is True + + +def test_classify_allows_a_checkout_hidden_behind_an_unrelated_dynamic_word() -> None: + """End-to-end regression pin for the round-11 finding at the + `classify()` level.""" + verdict = checker.classify("EDITOR=vim; $EDITOR sub; git checkout -- f.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("f.py",) + + # --- End-to-end classify() coverage, pinning every explicit safe/deny case # issue #1375's own Acceptance Criteria Map and "Explicit safe cases" # section name by hand. From f542d88a40f53bb30e13a90cef0af78d018a71d6 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 14:23:30 +0000 Subject: [PATCH 17/46] fix(hooks): fail closed when a resolved cd-relocator candidate is still dynamic A fresh, independent adversarial review of this PR's current head (round 12) found that round 11's own resolution primitive, _dynamic_word_may_resolve_to_a_cwd_relocator, never checked whether a value returned by _substitute_var_refs_candidates was itself still dynamic before comparing it against cd/pushd/popd -- unlike its sibling _resolve_path_tokens, which already carries this exact check for the identical reason. This matters because _substitute_var_refs_candidates does not recursively re-expand a ${NAME:-default} clause's own default text (a disclosed residual of that primitive itself): when the default text is itself a $OTHER reference, the returned candidate is the literal, still-unexpanded string "$OTHER", never equal to cd/pushd/popd as plain text even when $OTHER genuinely holds one of those at real bash runtime. Live-verified: OTHER=cd; ${UNSET:-$OTHER} sub; git checkout -- dirty.py resolved to a confident, wrong allow (checkout_restore_paths correctly extracted the filename, but the earlier segment's own cwd-relocation risk went undetected), and the real command silently discarded a genuinely dirty dirty.py -- the same failure class every earlier round in this PR has closed for a different gap. Fixed by adding the identical any(_is_dynamic(candidate) for candidate in candidates) fail-closed check _resolve_path_tokens already carries, treating a still-dynamic candidate as ambiguous/possibly-a-relocator rather than "not cd." Round 10's X=cd/X=pushd true positives and round 11's EDITOR=vim false-positive fix are both unaffected. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 25 ++++++++++++--- hooks/test_gitapex_check_bash_safety.py | 21 ++++++++++++ ...st_gitapex_check_bash_safety_properties.py | 32 +++++++++++++++++++ 3 files changed, 73 insertions(+), 5 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index c1d70520..1dde301f 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3626,7 +3626,7 @@ def _dynamic_word_may_resolve_to_a_cwd_relocator(token: str, name_to_raw_value: as literal, non-existent command `CD`, not the `cd` builtin) a false positive of its own. - Three cases: + Four cases: - No `$NAME`-shaped reference found in TOKEN at all (`_VAR_REF_FULL_RE.search` finds nothing) -- TOKEN's dynamism comes from something this resolution primitive cannot decompose (e.g. a @@ -3638,13 +3638,28 @@ def _dynamic_word_may_resolve_to_a_cwd_relocator(token: str, name_to_raw_value: readings to enumerate) or `[]` (some referenced name has no assigned-and-in-range reading this classifier can resolve) -- both genuine ambiguity, not a resolved-safe value. Fails closed (`True`). - - A concrete candidate list -- flags (`True`) only if some candidate - is exactly `cd`/`pushd`/`popd`; otherwise the word demonstrably - resolves to something else, so returns `False`.""" + - A returned candidate is ITSELF still dynamic (contains `$`/backtick + after substitution) -- CRITICAL bypass found by independent + adversarial review (round 12, issue #1375) and independently + reproduced live: `_substitute_var_refs_candidates` does NOT + recursively re-expand a `${NAME:-default}` clause's own DEFAULT + text (a disclosed residual of that primitive itself, see its own + docstring), so `OTHER=cd; ${UNSET:-$OTHER} sub; git checkout -- + dirty.py` resolved `${UNSET:-$OTHER}`'s one candidate to the + literal, still-unexpanded string `"$OTHER"` -- never equal to + `cd`/`pushd`/`popd` as plain text, even though `$OTHER` genuinely + holds `cd` at real bash runtime -- silently discarding uncommitted + work exactly like round 10's own original bypass. Fails closed + (`True`), mirroring the identical still-dynamic-candidate check + `_resolve_path_tokens` already carries for the same reason (see its + own docstring). + - A concrete, fully-resolved candidate list -- flags (`True`) only if + some candidate is exactly `cd`/`pushd`/`popd`; otherwise the word + demonstrably resolves to something else, so returns `False`.""" if _VAR_REF_FULL_RE.search(token) is None: return True candidates = _substitute_var_refs_candidates(token, name_to_raw_value, name_to_raw_value) - if candidates is None or not candidates: + if candidates is None or not candidates or any(_is_dynamic(candidate) for candidate in candidates): return True return any(candidate in _CWD_RELOCATING_COMMANDS for candidate in candidates) diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 997e37b0..779345d2 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1702,6 +1702,27 @@ def test_checkout_allowed_when_an_earlier_dynamic_word_resolves_to_something_har assert result.returncode == 0, f"stderr={result.stderr!r}" +def test_checkout_denied_when_an_earlier_default_clause_could_resolve_to_cd(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-12 independent review, issue + #1375). `_substitute_var_refs_candidates` does not recursively + re-expand a `${NAME:-default}` clause's own DEFAULT text -- so when + the default text is itself a `$OTHER` reference, the classifier's own + resolution returned the literal, still-unexpanded string `"$OTHER"`, + never equal to `cd`/`pushd`/`popd` as plain text even when `$OTHER` + genuinely holds one of those at real bash runtime. Live-verified + before this fix: `OTHER=cd; ${UNSET:-$OTHER} sub; git checkout -- + dirty.py` was wrongly allowed outright, and the real command silently + discarded a genuinely dirty `dirty.py`. The deny here is + classifier-level (a token-shape fact, no live git call), so no such + file needs to actually exist for this regression pin.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("OTHER=cd; ${UNSET:-$OTHER} sub; git checkout -- dirty.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 4a7799df..5f14ac6d 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3648,6 +3648,38 @@ def test_classify_allows_a_checkout_hidden_behind_an_unrelated_dynamic_word() -> assert verdict.checkout_restore_paths == ("f.py",) +def test_dynamic_word_may_resolve_to_a_cwd_relocator_true_for_a_still_dynamic_candidate() -> None: + """CRITICAL bypass regression pin (round-12 independent review, issue + #1375). `_substitute_var_refs_candidates` does NOT recursively + re-expand a `${NAME:-default}` clause's own DEFAULT text (a + disclosed residual of that primitive itself) -- so when the default + text is itself a `$OTHER` reference, the one returned candidate is + the literal, still-unexpanded string `"$OTHER"`, never equal to + `cd`/`pushd`/`popd` as plain text even when `$OTHER` genuinely holds + one of those at real bash runtime. Live-verified before this fix: + `${UNSET:-$OTHER}` with `OTHER=cd` resolved to a false `False` + (not-a-relocator) verdict instead of failing closed, mirroring the + identical still-dynamic-candidate check `_resolve_path_tokens` + already carries for the same reason.""" + assert checker._dynamic_word_may_resolve_to_a_cwd_relocator("${UNSET:-$OTHER}", {"OTHER": "cd"}) is True + assert checker._dynamic_word_may_resolve_to_a_cwd_relocator("${UNSET:-$OTHER}", {"OTHER": "pushd"}) is True + + +def test_rule_git_checkout_restore_denies_a_still_dynamic_candidate() -> None: + segments = [["${UNSET:-$OTHER}", "sub"], ["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}) + assert reason is not None + assert resolved == () + + +def test_classify_denies_a_checkout_hidden_behind_a_still_dynamic_default_clause() -> None: + """End-to-end regression pin for the round-12 finding at the + `classify()` level.""" + verdict = checker.classify("OTHER=cd; ${UNSET:-$OTHER} sub; git checkout -- dirty.py") + assert verdict.deny is True + assert verdict.checkout_restore_paths == () + + # --- End-to-end classify() coverage, pinning every explicit safe/deny case # issue #1375's own Acceptance Criteria Map and "Explicit safe cases" # section name by hand. From 29141659aa7d74923ae983a713f4ae98e0f578f5 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 14:40:07 +0000 Subject: [PATCH 18/46] fix(hooks): check the first surviving segment word, not seg[0] itself A fresh, independent adversarial review of this PR's current head (round 13) found that the cwd-relocation check inspected only a segment's raw seg[0] for a possible dynamic cd/pushd/popd -- when seg[0] itself genuinely vanishes at real bash runtime (e.g. a bare reference to a name never assigned), the whole check silently skipped that segment, even though the token that actually survives to become bash's real command word was never itself checked. This is a different code path than rounds 10-12 touched: it sits in the vanishing pre-check that runs before the dynamic-word resolution logic those rounds fixed, not in that resolution logic itself. Live-verified: X=cd; $NEVERSET $X sub; git checkout -- dirty.py (NEVERSET genuinely never assigned) resolved to a confident, wrong allow, and the real command genuinely runs `cd sub` there (confirmed against real bash via an argv-capturing cd proxy) before silently discarding a genuinely dirty dirty.py -- the same failure class every earlier round in this area has closed for a different gap. Same result for the pushd variant and for a vanishing ${NEVERSET:-} decoy in place of the bare reference. Fixed by adding _first_surviving_segment_word, which strips a leading run of vanishing tokens (per the same _token_is_all_unassigned_refs/ _token_is_a_vanishing_default_or_alt_clause checks this function already trusts) and returns the token that actually survives -- the same "collapse a leading vanishing run before re-checking the first position" pattern _classify_tokens's own collapsed_segments pass already applies elsewhere in this module via _strip_leading_unassigned_bare_refs. The dynamic-word check now runs against that surviving token instead of seg[0] directly; the literal scan (which already checks every token in the segment regardless of position) is unaffected. Rounds 10-12's own true positives and false-positive fixes are all unchanged. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 72 ++++++++++++++++--- hooks/test_gitapex_check_bash_safety.py | 21 ++++++ ...st_gitapex_check_bash_safety_properties.py | 40 +++++++++++ 3 files changed, 125 insertions(+), 8 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 1dde301f..4486934d 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3664,6 +3664,48 @@ def _dynamic_word_may_resolve_to_a_cwd_relocator(token: str, name_to_raw_value: return any(candidate in _CWD_RELOCATING_COMMANDS for candidate in candidates) +def _first_surviving_segment_word(seg: list[str], name_to_raw_value: dict[str, str]) -> str | None: + """The first token of SEG that would actually survive as bash's real + command word once every LEADING vanishing decoy -- a bare/braced + unassigned reference (`_token_is_all_unassigned_refs`) or an + empty-default/alt-value clause (`_token_is_a_vanishing_default_or_ + alt_clause`) -- has word-split away to nothing. `None` if the whole + leading run (up to and including every token in SEG) vanishes. + + CRITICAL bug found by independent adversarial review (round 13, issue + #1375) and independently reproduced live: `_rule_git_checkout_restore` + only ever checked `seg[0]` itself for a possible cwd-relocator, the + same way `_find_git_checkout_restore` checks `seg[0]` for the git/ + subcommand-position question -- but when `seg[0]` genuinely vanishes, + the classifier already knows (per this module's own established + convention, e.g. `_strip_leading_unassigned_bare_refs`'s use in + `_classify_tokens`'s own `collapsed_segments` pass) that whatever + token follows becomes the REAL command word at real bash runtime, and + that word was never itself checked here. Confirmed live: `X=cd; + $NEVERSET $X sub; git checkout -- dirty.py` (`NEVERSET` genuinely + never assigned) -- `$NEVERSET` vanishes, so the previous `seg[0]`-only + check silently skipped the whole segment, even though `$X` (which + resolves to `cd`) is what bash actually runs first. Real bash + (confirmed via an argv-capturing `cd` proxy) genuinely executes `cd + sub` there. Same result for `X=pushd` and for a vanishing + `${NEVERSET:-}` decoy in place of the bare `$NEVERSET`. + + Callers should feed the RESULT of this function to + `_dynamic_word_may_resolve_to_a_cwd_relocator` (when dynamic) or the + existing literal `_CWD_RELOCATING_COMMANDS` membership check (when + not), exactly like they would have used `seg[0]` directly before this + fix -- this function only changes WHICH token that check runs + against, never the check itself.""" + i = 0 + n = len(seg) + while i < n and ( + _token_is_all_unassigned_refs(seg[i], name_to_raw_value) + or _token_is_a_vanishing_default_or_alt_clause(seg[i], name_to_raw_value) + ): + i += 1 + return seg[i] if i < n else None + + def _rule_git_checkout_restore( segments: list[list[str]], raw_assigned: dict[str, str] ) -> tuple[str | None, tuple[str, ...]]: @@ -3730,20 +3772,34 @@ def _rule_git_checkout_restore( docstring) and only flags when a candidate could genuinely be `cd`/`pushd`/`popd`, or resolution is itself ambiguous/unresolvable -- never widening beyond what round 10 already flagged, only narrowing - it.""" + it. + + The dynamic-word check above runs against `_first_surviving_segment_ + word(seg, raw_assigned)`, not `seg[0]` directly -- CRITICAL bug found + by independent adversarial review (round 13, issue #1375) and + independently reproduced live: a `seg[0]`-only check silently skips + the whole segment when `seg[0]` itself genuinely vanishes, even + though the token that actually survives to become bash's real command + word (per that same vanishing logic this function already trusts + elsewhere) was never itself checked. `X=cd; $NEVERSET $X sub; git + checkout -- dirty.py` (`NEVERSET` genuinely never assigned) resolved + to the same CONFIDENT, WRONG `checkout_restore_paths` claim every + earlier round in this area has closed for a different gap -- see + `_first_surviving_segment_word`'s own docstring for the full + reproduction. The literal scan above is unaffected: it already checks + every token in the segment regardless of position, so a literal + `cd`/`pushd`/`popd` sitting after a vanishing decoy was already + covered.""" saw_cd = False all_paths: list[str] = [] for seg in segments: subcommand, tokens_after, saw_tree_relocation = _find_git_checkout_restore(seg, raw_assigned) if subcommand is None: + first = _first_surviving_segment_word(seg, raw_assigned) if any(not _is_dynamic(t) and t in _CWD_RELOCATING_COMMANDS for t in seg) or ( - seg - and _is_dynamic(seg[0]) - and not ( - _token_is_all_unassigned_refs(seg[0], raw_assigned) - or _token_is_a_vanishing_default_or_alt_clause(seg[0], raw_assigned) - ) - and _dynamic_word_may_resolve_to_a_cwd_relocator(seg[0], raw_assigned) + first is not None + and _is_dynamic(first) + and _dynamic_word_may_resolve_to_a_cwd_relocator(first, raw_assigned) ): saw_cd = True continue diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 779345d2..8991d997 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1723,6 +1723,27 @@ def test_checkout_denied_when_an_earlier_default_clause_could_resolve_to_cd(tmp_ assert "working tree is at risk" in payload["systemMessage"] +def test_checkout_denied_when_a_dynamic_relocator_sits_behind_a_leading_vanishing_decoy(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-13 independent review, issue + #1375). The classifier's own cwd-relocation check only ever inspected + a segment's first token -- when that first token itself genuinely + vanishes at real bash runtime (e.g. a bare reference to a name never + assigned), the token that actually survives to become the real + command word was never itself checked. Live-verified before this fix: + `X=cd; $NEVERSET $X sub; git checkout -- dirty.py` (`NEVERSET` + genuinely never assigned) was wrongly allowed outright, and the real + command silently discarded a genuinely dirty `dirty.py` -- real bash + genuinely runs `cd sub` there. The deny here is classifier-level (a + token-shape fact, no live git call), so no such file needs to + actually exist for this regression pin.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("X=cd; $NEVERSET $X sub; git checkout -- dirty.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 5f14ac6d..4d0085fa 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3680,6 +3680,46 @@ def test_classify_denies_a_checkout_hidden_behind_a_still_dynamic_default_clause assert verdict.checkout_restore_paths == () +def test_first_surviving_segment_word_skips_a_leading_vanishing_run() -> None: + """A leading run of vanishing decoys (bare-unassigned, then an empty + default clause) is skipped, landing on the real surviving word -- + whether that word is dynamic or a plain literal.""" + assert checker._first_surviving_segment_word(["$NEVERSET", "${OTHER:-}", "$X"], {"X": "cd"}) == "$X" + assert checker._first_surviving_segment_word(["$NEVERSET", "sub"], {}) == "sub" + + +def test_first_surviving_segment_word_none_when_everything_vanishes() -> None: + assert checker._first_surviving_segment_word(["$NEVERSET", "${OTHER:-}"], {}) is None + + +def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_vanishing_decoy() -> None: + """CRITICAL bypass regression pin (round-13 independent review, issue + #1375). The prior `seg[0]`-only check silently skipped the whole + segment whenever `seg[0]` itself genuinely vanished, even though the + token that actually survives to become bash's real command word was + never itself checked. Live-verified before this fix: `X=cd; $NEVERSET + $X sub; git checkout -- dirty.py` (`NEVERSET` genuinely never + assigned) resolved to a CONFIDENT, WRONG `checkout_restore_paths` + claim -- real bash genuinely runs `cd sub` there.""" + segments = [["$NEVERSET", "$X", "sub"], ["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}) + assert reason is not None + assert resolved == () + + +def test_classify_denies_a_checkout_hidden_behind_a_leading_vanishing_decoy() -> None: + """End-to-end regression pin for the round-13 finding at the + `classify()` level.""" + for cmd in ( + "X=cd; $NEVERSET $X sub; git checkout -- dirty.py", + "X=pushd; $NEVERSET $X sub; git checkout -- dirty.py", + "X=cd; ${NEVERSET:-} $X sub; git checkout -- dirty.py", + ): + verdict = checker.classify(cmd) + assert verdict.deny is True, cmd + assert verdict.checkout_restore_paths == () + + # --- End-to-end classify() coverage, pinning every explicit safe/deny case # issue #1375's own Acceptance Criteria Map and "Explicit safe cases" # section name by hand. From d109eba0ccd03afb36efb1d6c44a76805f1c8346 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 15:07:16 +0000 Subject: [PATCH 19/46] fix(hooks): recognize a redirect clause and a dynamic git token A fresh, independent adversarial review of this PR's current head (round 14) found three related gaps, all in the same checkout/restore detection area: 1. An I/O-redirection clause (`>`/`<`/`>>`/etc.) sitting before a dynamic command word broke _first_surviving_segment_word's leading- decoy-skip walk: the redirect operator token itself (neither vanishing nor dynamic) was returned as the "surviving word," leaving the real, cd-resolving token one position later unchecked. Live-verified: X=cd; > /dev/null $X sub; git checkout -- dirty.py resolved to a confident, wrong ALLOW. 2. The same redirect shape sitting between a literal git and its subcommand broke _find_git_checkout_restore's own flag-skip loop the identical way, for a fully literal command with no dynamic content at all. Live-verified: git > /dev/null checkout -- dirty.py resolved to an empty, wrong checkout_restore_paths, so the wrapper's live git-diff check never even ran. 3. Only a LITERAL git token was ever recognized as the start of a checkout/restore invocation. Live-verified: G=git; $G checkout -- dirty.py resolved to an empty, wrong checkout_restore_paths even though $G unambiguously resolves to git. All three were live-verified end-to-end through the real hooks/check-bash-safety.sh wrapper against a scratch git repo to genuinely, silently discard a dirty file when actually executed. Fixed with two additions, both reused across the affected call sites: - _redirect_span_length recognizes a redirect clause (an optional leading bare fd number, a redirect operator, and its target token -- segment_tokens never splits a segment at , so each survives as its own token) and reports how many tokens to skip. _first_surviving_segment_word and _find_git_checkout_restore's flag-skip loop both now skip a redirect clause wherever they would otherwise stop on one, closing findings 1 and 2. - _dynamic_token_resolves_only_to_literal resolves a dynamic token's candidate value(s) via the existing _substitute_var_refs_candidates primitive and requires every candidate to match the target literal -- an ambiguous or unresolvable token declines rather than assumes the positive case, since a false positive here would mis-attribute an unrelated tool's own subcommand (e.g. $TOOL checkout where TOOL is not git) as a git invocation. _find_git_checkout_restore's own outer scan now also recognizes a dynamic token resolving only to git, closing finding 3, without widening what it flags for any unresolvable or unrelated dynamic first word. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 145 +++++++++++++++++- hooks/test_gitapex_check_bash_safety.py | 50 ++++++ ...st_gitapex_check_bash_safety_properties.py | 132 ++++++++++++++++ 3 files changed, 319 insertions(+), 8 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 4486934d..310fe755 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3505,6 +3505,89 @@ def _token_is_a_vanishing_default_or_alt_clause(token: str, name_to_raw_value: d return False +_REDIRECT_OPERATORS = {"<", ">", ">>", "<<", "<<<", "&>", ">&", "&>>", "<>"} + + +def _redirect_span_length(seg: list[str], j: int) -> int: + """The number of tokens, starting at SEG[j], that make up one bash + I/O-redirection clause -- an optional leading bare file-descriptor + number (`2>`), a redirect operator (`_REDIRECT_OPERATORS`), and + exactly one target token. `segment_tokens` never splits a segment at + `<`/`>` (see `_SINGLE_OPS`'s own docstring -- a redirect may legally + sit anywhere before a command word without being that word itself), + so each piece of the clause survives here as its own separate token + (confirmed via `tokenize()`: `git > /dev/null checkout` tokenizes to + `['git', '>', '/dev/null', 'checkout']`; `git 2> /dev/null checkout` + to `['git', '2', '>', '/dev/null', 'checkout']`). Returns 0 when + SEG[j:] does not start with this shape, so a caller can treat 0 as + "not a redirect, do not skip" without a separate boolean check. + + CRITICAL bug found by independent adversarial review (round 14, issue + #1375) and independently reproduced live: every existing skip-past- + decoy walk in this file's checkout/restore detection -- + `_find_git_checkout_restore`'s own global-flag-skip loop and + `_first_surviving_segment_word`'s own leading-vanishing-run walk -- + had no concept of a redirect clause at all, so a bare `>`/`<` token + (ordinary, legal bash syntax) broke both: `git > /dev/null checkout + -- dirty.py` (a fully literal command, no dynamic content at all) + resolved to an empty, wrong `checkout_restore_paths` (the redirect + operator token itself was mistaken for the subcommand position and + the scan gave up), and `X=cd; > /dev/null $X sub; git checkout -- + dirty.py` resolved to a confident, wrong ALLOW (the redirect made + `_first_surviving_segment_word` return the operator token itself, + which is neither vanishing nor dynamic, so the real, cd-resolving + `$X` one position later was never checked) -- both live-verified + (real bash, and end-to-end through the real wrapper against a + scratch git repo) to silently discard a genuinely dirty file.""" + n = len(seg) + i = j + if i < n and seg[i].isdigit(): + i += 1 + if i < n and seg[i] in _REDIRECT_OPERATORS and i + 1 < n: + return i + 2 - j + return 0 + + +def _dynamic_token_resolves_only_to_literal(token: str, name_to_raw_value: dict[str, str], literal: str) -> bool: + """Whether TOKEN unambiguously resolves, at real bash runtime, to + exactly LITERAL (case-insensitively, matching this function's own + caller's existing case-insensitive literal comparison) and nothing + else -- narrower than "could plausibly resolve to LITERAL": an + ambiguous or unresolvable token declines (returns `False`) rather + than assuming the positive case, since a false positive here would + mis-attribute an unrelated dynamic command word's own subcommand + (e.g. `$TOOL checkout` where TOOL is some other, non-git tool that + also happens to have a `checkout` subcommand) as a git checkout/ + restore invocation -- unlike the cwd-relocation check's own + fail-closed posture, `_find_git_checkout_restore`'s own docstring + already establishes that an ambiguous "is this actually git" question + here declines to resolve rather than assumes the worst (see its own + "genuinely ambiguous... this pure classifier declines to resolve" + paragraph, for the analogous ambiguous-token-after-`git` case this + mirrors for the `git` token itself). + + CRITICAL bug found by independent adversarial review (round 14, issue + #1375) and independently reproduced live: `_find_git_checkout_ + restore`'s own outer scan only ever recognized a LITERAL `git` token + -- `G=git; $G checkout -- dirty.py` resolved to an empty, wrong + `checkout_restore_paths` even though `$G` unambiguously resolves to + `git`. Live-verified this genuinely runs `git checkout -- dirty.py` + once bash resolves it. Reuses `_substitute_var_refs_candidates` + exactly like `_dynamic_word_may_resolve_to_a_cwd_relocator` does for + the analogous cd/pushd/popd question, but requires EVERY candidate + reading to match LITERAL (not just one), the mirror-image of that + function's OR-based check -- appropriate here because a false + positive in THIS position risks a wrong `checkout_restore_paths` + CLAIM about an unrelated tool, while that function's own false + positive would only over-deny (the safer direction).""" + if _VAR_REF_FULL_RE.search(token) is None: + return False + candidates = _substitute_var_refs_candidates(token, name_to_raw_value, name_to_raw_value) + if candidates is None or not candidates: + return False + return all(candidate.lower() == literal for candidate in candidates) + + def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str]) -> tuple[str | None, list[str], bool]: """Scan SEG (already assignment-stripped, see `_strip_leading_ assignments`) for a `git checkout`/`git restore` invocation, skipping @@ -3555,10 +3638,35 @@ def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str] `KNOWN_BYPASS_COMMANDS` test list already uses for the analogous dynamic-tool/dynamic-verb case; scanning continues past it to any later `git` occurrence in the same segment), in which case the other - two return values are meaningless.""" + two return values are meaningless. + + A DYNAMIC `tok` that unambiguously resolves to `git` (per + `_dynamic_token_resolves_only_to_literal`, see its own docstring) is + ALSO recognized as this `git` occurrence -- CRITICAL bug found by + independent adversarial review (round 14, issue #1375) and + independently reproduced live: `G=git; $G checkout -- dirty.py` + resolved to an empty, wrong `checkout_restore_paths` before this fix, + even though `$G` unambiguously resolves to `git` and this genuinely + runs `git checkout -- dirty.py` once bash resolves it. An ambiguous + or unresolvable dynamic `tok` still declines here (returns to the + outer scan without treating it as `git`), the same "decline, don't + assume" posture already established above for an ambiguous token + AFTER a literal `git`. + + The flag-skip loop below also skips a redirect clause + (`_redirect_span_length`, see its own docstring) wherever it would + otherwise land -- CRITICAL bug found by independent adversarial + review (round 14, issue #1375) and independently reproduced live: + `git > /dev/null checkout -- dirty.py` (fully literal, no dynamic + content at all) resolved to an empty, wrong `checkout_restore_paths` + before this fix, since the redirect operator token itself was + mistaken for the subcommand position and the scan gave up there.""" n = len(seg) for i, tok in enumerate(seg): - if _is_dynamic(tok) or tok.lower() != "git": + if _is_dynamic(tok): + if not _dynamic_token_resolves_only_to_literal(tok, name_to_raw_value, "git"): + continue + elif tok.lower() != "git": continue saw_tree_relocation = False j = i + 1 @@ -3573,6 +3681,10 @@ def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str] continue ambiguous = True break + redirect_len = _redirect_span_length(seg, j) + if redirect_len: + j += redirect_len + continue if ( candidate == "-C" or candidate in _GIT_TREE_RELOCATION_LONG_FLAGS @@ -3695,14 +3807,31 @@ def _first_surviving_segment_word(seg: list[str], name_to_raw_value: dict[str, s existing literal `_CWD_RELOCATING_COMMANDS` membership check (when not), exactly like they would have used `seg[0]` directly before this fix -- this function only changes WHICH token that check runs - against, never the check itself.""" + against, never the check itself. + + Also skips a leading redirect clause (`_redirect_span_length`, see + its own docstring) -- CRITICAL bug found by independent adversarial + review (round 14, issue #1375) and independently reproduced live: + `X=cd; > /dev/null $X sub; git checkout -- dirty.py` resolved to a + confident, wrong ALLOW before this fix, since the redirect made this + walk return the `>` operator token itself (neither vanishing nor + dynamic) as the "surviving word," so the real, cd-resolving `$X` one + position later was never checked. A vanishing decoy and a redirect + clause may interleave in either order (`$NEVERSET > /dev/null $X + sub`), so both skips run in the SAME loop until neither applies.""" i = 0 n = len(seg) - while i < n and ( - _token_is_all_unassigned_refs(seg[i], name_to_raw_value) - or _token_is_a_vanishing_default_or_alt_clause(seg[i], name_to_raw_value) - ): - i += 1 + while i < n: + if _token_is_all_unassigned_refs(seg[i], name_to_raw_value) or _token_is_a_vanishing_default_or_alt_clause( + seg[i], name_to_raw_value + ): + i += 1 + continue + redirect_len = _redirect_span_length(seg, i) + if redirect_len: + i += redirect_len + continue + break return seg[i] if i < n else None diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 8991d997..94cf49f7 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1744,6 +1744,56 @@ def test_checkout_denied_when_a_dynamic_relocator_sits_behind_a_leading_vanishin assert "working tree is at risk" in payload["systemMessage"] +def test_checkout_denied_when_a_dynamic_relocator_sits_behind_a_leading_redirect(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-14 independent review, issue + #1375). A leading I/O-redirection clause -- ordinary, legal bash + syntax -- made the classifier's own leading-decoy-skip walk return + the redirect operator token itself as the "surviving word," so the + real, cd-resolving `$X` one position later was never checked. + Live-verified before this fix: `X=cd; > /dev/null $X sub; git + checkout -- dirty.py` was wrongly allowed outright, and the real + command silently discarded a genuinely dirty `dirty.py`. The deny + here is classifier-level (a token-shape fact, no live git call), so + no such file needs to actually exist for this regression pin.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("X=cd; > /dev/null $X sub; git checkout -- dirty.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + +def test_checkout_denied_when_a_redirect_sits_between_git_and_the_subcommand(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-14 independent review, issue + #1375). A fully literal command -- no dynamic content at all -- with + a redirect between `git` and its subcommand was invisible to + detection entirely: the redirect operator token was mistaken for the + subcommand position and the scan gave up, so `checkout_restore_paths` + resolved empty and the live wrapper check never even ran. + Live-verified before this fix: `git > /dev/null checkout -- dirty.py` + was wrongly allowed outright, and the real command silently discarded + a genuinely dirty `dirty.py`.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("git > /dev/null checkout -- dirty.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_checkout_denied_when_git_itself_is_a_dynamic_word(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-14 independent review, issue + #1375). Only a LITERAL `git` token was ever recognized as the start + of a checkout/restore invocation -- live-verified before this fix: + `G=git; $G checkout -- dirty.py` was wrongly allowed outright, even + though `$G` unambiguously resolves to `git`, and the real command + silently discarded a genuinely dirty `dirty.py`.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("G=git; $G checkout -- dirty.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 4d0085fa..03b7b8ff 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3720,6 +3720,138 @@ def test_classify_denies_a_checkout_hidden_behind_a_leading_vanishing_decoy() -> assert verdict.checkout_restore_paths == () +def test_redirect_span_length_recognizes_operator_and_target() -> None: + assert checker._redirect_span_length([">", "/dev/null", "x"], 0) == 2 + assert checker._redirect_span_length(["2", ">", "/dev/null", "x"], 0) == 3 + assert checker._redirect_span_length(["2", ">&", "1", "x"], 0) == 3 + + +def test_redirect_span_length_zero_when_no_redirect_present() -> None: + assert checker._redirect_span_length(["checkout", "--", "f.py"], 0) == 0 + assert checker._redirect_span_length(["2", "checkout", "--", "f.py"], 0) == 0 + assert checker._redirect_span_length([">"], 0) == 0 + + +def test_first_surviving_segment_word_skips_a_leading_redirect() -> None: + assert checker._first_surviving_segment_word([">", "/dev/null", "$X"], {"X": "cd"}) == "$X" + assert checker._first_surviving_segment_word(["$NEVERSET", ">", "/dev/null", "$X"], {"X": "cd"}) == "$X" + + +def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_redirect() -> None: + """CRITICAL bypass regression pin (round-14 independent review, issue + #1375). A leading redirect clause (ordinary, legal bash syntax) made + `_first_surviving_segment_word` return the redirect operator token + itself -- neither vanishing nor dynamic -- as the "surviving word," + so the real, cd-resolving `$X` one position later was never checked. + Live-verified before this fix: `X=cd; > /dev/null $X sub; git + checkout -- dirty.py` resolved to a confident, wrong ALLOW.""" + segments = [[">", "/dev/null", "$X", "sub"], ["git", "checkout", "--", "f.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}) + assert reason is not None + assert resolved == () + + +def test_classify_denies_a_checkout_hidden_behind_a_leading_redirect() -> None: + """End-to-end regression pin for the round-14 redirect-before-dynamic- + word finding at the `classify()` level.""" + verdict = checker.classify("X=cd; > /dev/null $X sub; git checkout -- dirty.py") + assert verdict.deny is True + assert verdict.checkout_restore_paths == () + + +def test_find_git_checkout_restore_skips_a_redirect_between_git_and_subcommand() -> None: + """CRITICAL bypass regression pin (round-14 independent review, issue + #1375). A fully literal command with a redirect between `git` and its + subcommand was invisible to detection -- the redirect operator token + was mistaken for the subcommand position and the scan gave up. + Live-verified before this fix: `git > /dev/null checkout -- dirty.py` + resolved to an empty, wrong `checkout_restore_paths`.""" + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore( + ["git", ">", "/dev/null", "checkout", "--", "f.py"], {} + ) + assert subcommand == "checkout" + assert tokens_after == ["--", "f.py"] + assert saw_tree_relocation is False + + +def test_classify_extracts_paths_behind_a_redirect_between_git_and_subcommand() -> None: + """End-to-end regression pin for the round-14 redirect-between-git- + and-subcommand finding at the `classify()` level.""" + verdict = checker.classify("git > /dev/null checkout -- dirty.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_VALUES) +def test_dynamic_token_resolves_only_to_literal_matches_the_resolved_value(name: str, value: str) -> None: + """Model-based: `_dynamic_token_resolves_only_to_literal` returns + `True` if and only if the token's one resolved candidate equals the + target literal, case-insensitively.""" + result = checker._dynamic_token_resolves_only_to_literal(f"${name}", {name: value}, "git") + assert result == (value.lower() == "git") + + +def test_dynamic_token_resolves_only_to_literal_true_for_an_unambiguous_match() -> None: + assert checker._dynamic_token_resolves_only_to_literal("$G", {"G": "git"}, "git") is True + assert checker._dynamic_token_resolves_only_to_literal("$G", {"G": "GIT"}, "git") is True + + +def test_dynamic_token_resolves_only_to_literal_false_for_an_unrelated_value() -> None: + assert checker._dynamic_token_resolves_only_to_literal("$G", {"G": "svn"}, "git") is False + + +def test_dynamic_token_resolves_only_to_literal_false_when_unresolvable() -> None: + """A false positive here would mis-attribute an unrelated tool's own + subcommand (e.g. `$TOOL checkout` where TOOL is not git) as a git + checkout/restore invocation, so an ambiguous or unresolvable token + declines rather than assumes the positive case -- the mirror image of + `_dynamic_word_may_resolve_to_a_cwd_relocator`'s own fail-closed + posture, appropriate here because the risk direction is reversed.""" + assert checker._dynamic_token_resolves_only_to_literal("$UNKNOWN", {}, "git") is False + + +def test_find_git_checkout_restore_recognizes_a_dynamic_git_token() -> None: + """CRITICAL bypass regression pin (round-14 independent review, issue + #1375). Only a LITERAL `git` token was ever recognized as the start + of a checkout/restore invocation -- live-verified before this fix: + `G=git; $G checkout -- dirty.py` resolved to an empty, wrong + `checkout_restore_paths` even though `$G` unambiguously resolves to + `git`.""" + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore( + ["$G", "checkout", "--", "f.py"], {"G": "git"} + ) + assert subcommand == "checkout" + assert tokens_after == ["--", "f.py"] + assert saw_tree_relocation is False + + +def test_find_git_checkout_restore_declines_an_unresolvable_dynamic_first_word() -> None: + """No false positive: a dynamic first token that does not unambiguously + resolve to `git` (unrelated tool, or unresolvable) is not mistaken for + a git invocation.""" + subcommand, _tokens_after, _saw_tree_relocation = checker._find_git_checkout_restore( + ["$TOOL", "checkout", "--", "f.py"], {"TOOL": "svn"} + ) + assert subcommand is None + + +def test_classify_extracts_paths_behind_a_dynamic_git_token() -> None: + """End-to-end regression pin for the round-14 dynamic-git-token + finding at the `classify()` level.""" + verdict = checker.classify("G=git; $G checkout -- dirty.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + +def test_classify_does_not_extract_paths_from_an_unrelated_dynamic_tool() -> None: + """No false positive: `$TOOL checkout -- x` where TOOL resolves to a + non-git tool must not be mistaken for a git checkout.""" + verdict = checker.classify("TOOL=svn; $TOOL checkout -- x") + assert verdict.deny is False + assert verdict.checkout_restore_paths == () + + # --- End-to-end classify() coverage, pinning every explicit safe/deny case # issue #1375's own Acceptance Criteria Map and "Explicit safe cases" # section name by hand. From 929f42cf5ca76e8419c4e51240ba114685c98664 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 15:30:31 +0000 Subject: [PATCH 20/46] fix(hooks): exclude a redirect clause from extracted checkout/restore paths A fresh, independent adversarial review of this PR's current head (round 15) found that round 14's own redirect-awareness only reached the functions that find the checkout/restore invocation (_find_git_checkout_restore, _first_surviving_segment_word), never the functions that extract paths from it once found (_git_checkout_paths, _git_restore_paths, both funneling through _resolve_path_tokens). A redirect operator and its target token were swept into checkout_restore_paths as if they were real git path arguments, since neither ever starts with "-" and both slipped past every existing positional/path filter unchanged. Live-verified: git checkout -- f.py >> unrelated_append_target.py resolved to checkout_restore_paths=('f.py', '>>', 'unrelated_append_target.py') -- a confident, wrong claim. Confirmed end-to-end through the real hooks/check-bash-safety.sh wrapper against a scratch git repo: whenever unrelated_append_target.py happened to be genuinely dirty (even though an append redirect can never discard its existing content), the wrapper wrongly denied a checkout that provably never touches that file. Fixed by adding _strip_redirect_clauses, which strips every redirect clause from a token list using the existing _redirect_span_length primitive, and applying it once, up front, to the whole tokens_after in both _git_checkout_paths and _git_restore_paths, before any --split, positional-count, or flag walk runs. Applying it early rather than only inside _resolve_path_tokens matters: stripping late would still leave a redirect's own token count distorting _git_checkout_paths's own sub-case (b) decision (2+ real positionals with no --), since real git never sees the redirect's tokens as positionals at all. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 66 ++++++++++++++++++- hooks/test_gitapex_check_bash_safety.py | 20 ++++++ ...st_gitapex_check_bash_safety_properties.py | 43 ++++++++++++ 3 files changed, 127 insertions(+), 2 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 310fe755..4f90cfa0 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3298,7 +3298,14 @@ def _git_checkout_paths( this classifier simply cannot read the named file -- so silently granting no-claim safety here would under-serve the exact opaque-path threat model this whole feature exists to close, not merely decline - to extend coverage.""" + to extend coverage. + + A redirect clause (`_strip_redirect_clauses`, see its own docstring) + is stripped from TOKENS_AFTER FIRST, before any check below runs -- + CRITICAL bug found by independent adversarial review (round 15, issue + #1375): a redirect operator and its target were otherwise swept into + `checkout_restore_paths` as if they were real git path arguments.""" + tokens_after = _strip_redirect_clauses(tokens_after) if any(tok in _CHECKOUT_BRANCH_CREATION_FLAGS or tok.startswith("--orphan=") for tok in tokens_after): return None, () if any( @@ -3345,7 +3352,14 @@ def _git_restore_paths( including `--pathspec-from-file`/`--pathspec-file-nul`, whose paths come from a file this classifier cannot inspect -- denies outright rather than risk under-extracting paths past a flag whose own - value-consumption behavior is unknown here.""" + value-consumption behavior is unknown here. + + A redirect clause (`_strip_redirect_clauses`, see its own docstring) + is stripped from TOKENS_AFTER FIRST, before this walk runs -- CRITICAL + bug found by independent adversarial review (round 15, issue #1375): + a redirect operator and its target were otherwise swept into + `checkout_restore_paths` as if they were real git path arguments.""" + tokens_after = _strip_redirect_clauses(tokens_after) saw_staged = False saw_worktree = False path_tokens: list[str] = [] @@ -3548,6 +3562,54 @@ def _redirect_span_length(seg: list[str], j: int) -> int: return 0 +def _strip_redirect_clauses(tokens: list[str]) -> list[str]: + """TOKENS with every I/O-redirection clause (`_redirect_span_length`, + see its own docstring) removed, wherever it occurs -- the shell + consumes a redirect clause itself; it is never passed to the command + (here, `git checkout`/`git restore`) as one of its own arguments. + + CRITICAL bug found by independent adversarial review (round 15, issue + #1375) and independently reproduced live: round 14 taught + `_find_git_checkout_restore` (finding the `checkout`/`restore` word + itself) and `_first_surviving_segment_word` (finding a possible cwd + relocator) to skip a redirect clause, but never taught the PATH- + EXTRACTION functions (`_git_checkout_paths`, `_git_restore_paths`, + both of which call this on TOKENS_AFTER before doing anything else) + the same lesson -- so a redirect operator and its target token were + swept into `checkout_restore_paths` as if they were real git path + arguments. `git checkout -- f.py >> unrelated_append_target.py` + resolved to `checkout_restore_paths=('f.py', '>>', + 'unrelated_append_target.py')` -- a CONFIDENT, WRONG claim (redirect + operators never start with `-`, so every existing positional/path + filter in this section swept them in) that made the live wrapper + check deny a checkout that provably never touches + `unrelated_append_target.py` at all (an append redirect can only add + to a file, never discard its existing content) whenever that + unrelated file happened to be dirty. Confirmed live end-to-end + through the real wrapper against a scratch git repo: the flagged + file's content was byte-for-byte unchanged after the real command + actually ran. + + Applied ONCE, up front, to the whole TOKENS_AFTER in both callers -- + before any `--`-split, positional-count, or flag walk runs -- rather + than taught separately to `_resolve_path_tokens` and each caller's + own positional-gathering logic: stripping late would still leave a + redirect's own token COUNT distorting `_git_checkout_paths`'s own + sub-case (b) decision (2+ real positionals with no `--`), since real + git never sees the redirect's tokens as positionals at all.""" + result: list[str] = [] + i = 0 + n = len(tokens) + while i < n: + redirect_len = _redirect_span_length(tokens, i) + if redirect_len: + i += redirect_len + continue + result.append(tokens[i]) + i += 1 + return result + + def _dynamic_token_resolves_only_to_literal(token: str, name_to_raw_value: dict[str, str], literal: str) -> bool: """Whether TOKEN unambiguously resolves, at real bash runtime, to exactly LITERAL (case-insensitively, matching this function's own diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 94cf49f7..23fc4b98 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1467,6 +1467,26 @@ def test_checkout_allowed_when_a_comment_after_a_line_continuation_names_an_unre ) assert result.returncode == 0, f"stderr={result.stderr!r}" assert result.stdout == "" + + +def test_checkout_allowed_when_a_trailing_redirect_names_an_unrelated_dirty_file(tmp_path: Path) -> None: + """CRITICAL false-positive regression pin (round-15 independent + review, issue #1375). Round 14 taught `_find_git_checkout_restore` + and `_first_surviving_segment_word` to skip a redirect clause, but + never taught the path-extraction functions the same lesson -- a + redirect operator and its target were swept into + `checkout_restore_paths` as if they were real git path arguments. + The real checkout target (`f.py`) is untouched; `unrelated.log` is + dirty but only ever used as an append-redirect target, which can + never discard its existing content.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + (repo_dir / "unrelated.log").write_text("hello\n") + _git(repo_dir, "add", "unrelated.log") + _git(repo_dir, "commit", "-q", "-m", "add unrelated.log") + (repo_dir / "unrelated.log").write_text("hello\ndirty\n") + result = run("git checkout -- f.py >> unrelated.log", payload_cwd=str(repo_dir)) + assert result.returncode == 0, f"stderr={result.stderr!r}" assert result.stderr == "" diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 03b7b8ff..a7dad4bf 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3852,6 +3852,49 @@ def test_classify_does_not_extract_paths_from_an_unrelated_dynamic_tool() -> Non assert verdict.checkout_restore_paths == () +def test_strip_redirect_clauses_removes_a_redirect_wherever_it_occurs() -> None: + assert checker._strip_redirect_clauses(["f.py", ">>", "log.txt"]) == ["f.py"] + assert checker._strip_redirect_clauses([">>", "log.txt", "f.py"]) == ["f.py"] + assert checker._strip_redirect_clauses(["f.py", "2", ">&", "1"]) == ["f.py"] + assert checker._strip_redirect_clauses(["f.py", "g.py"]) == ["f.py", "g.py"] + + +def test_git_checkout_paths_excludes_a_trailing_redirect_clause() -> None: + """CRITICAL false-positive regression pin (round-15 independent + review, issue #1375). Round 14 taught `_find_git_checkout_restore` + and `_first_surviving_segment_word` to skip a redirect clause, but + never taught the path-extraction functions the same lesson -- a + redirect operator and its target were swept into + `checkout_restore_paths` as if they were real git path arguments. + Live-verified before this fix: `git checkout -- f.py >> + unrelated_append_target.py` resolved to `checkout_restore_paths= + ('f.py', '>>', 'unrelated_append_target.py')`, causing the live + wrapper check to wrongly deny whenever the unrelated append target + happened to be dirty, even though an append redirect can never + discard that file's existing content.""" + deny_reason, paths = checker._git_checkout_paths(["--", "f.py", ">>", "unrelated_append_target.py"], {}) + assert deny_reason is None + assert paths == ("f.py",) + + +def test_git_restore_paths_excludes_a_trailing_redirect_clause() -> None: + deny_reason, paths = checker._git_restore_paths(["f.py", ">>", "unrelated_append_target.py"], {}) + assert deny_reason is None + assert paths == ("f.py",) + + +def test_classify_does_not_flag_a_redirect_target_as_a_checkout_path() -> None: + """End-to-end regression pin for the round-15 finding at the + `classify()` level.""" + for cmd in ( + "git checkout -- f.py >> unrelated_append_target.py", + "git restore f.py >> unrelated_append_target.py", + ): + verdict = checker.classify(cmd) + assert verdict.deny is False, cmd + assert verdict.checkout_restore_paths == ("f.py",), cmd + + # --- End-to-end classify() coverage, pinning every explicit safe/deny case # issue #1375's own Acceptance Criteria Map and "Explicit safe cases" # section name by hand. From c08dfdef6644386c68096a27401fea4d3446ca86 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 15:48:08 +0000 Subject: [PATCH 21/46] fix(hooks): split strict vs lenient redirect-fd handling by call site A fresh, independent adversarial review of this PR's current head (round 16) found that _redirect_span_length's own leading-file- descriptor-number heuristic (added in round 14) was unsound. tokenize()'s own shlex punctuation-splitting cannot distinguish a fused 2>file (a genuine fd-redirect prefix, no argument) from a spaced 2 >file (the literal word 2 followed by a separate redirect) -- both produce the identical token sequence, so the whitespace adjacency that actually tells them apart is already gone by the time this function ever sees the tokens. The prior version always guessed "consumed by the redirect," the unsafe direction for the two path-extraction callers it feeds through _strip_redirect_clauses. Live-verified: git checkout -- realfile.py 2 >target.txt resolved to checkout_restore_paths=('realfile.py',), silently dropping 2 (a real, dirty, tracked file). Worse: git restore --source 2 >target.txt file.py resolved to an EMPTY checkout_restore_paths, since --source's own value-consumption then swallowed file.py itself once 2 vanished into the wrongly-recognized redirect -- both confirmed end-to-end through the real wrapper to silently discard a genuinely dirty file. Fixed by making _redirect_span_length itself strict (never consumes a leading digit), used by _strip_redirect_clauses for path extraction, where over-inclusion is the safe direction. Simply removing the digit handling everywhere regressed the subcommand-finding walks, though: without it, a fully literal git > out.log 2>&1 checkout -- dirty.py stopped being recognized as a checkout invocation at all (the bare digit token broke the flag-skip loop), leaving checkout_restore_paths empty and the live wrapper check never running -- the fail-open direction for that walk. Added a second variant, _redirect_span_length_with_optional_fd, which does consume an optional leading digit, and pointed the two skip-PAST-a-possible- redirect walks (_find_git_checkout_restore's flag-skip loop, _first_surviving_segment_word) at it instead -- the two contexts need opposite defaults for the same digit-adjacency ambiguity, since skipping too little there means missing a real invocation entirely, while skipping too much in path extraction means missing a real dirty path. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 96 ++++++++++++++++--- hooks/test_gitapex_check_bash_safety.py | 34 +++++++ ...st_gitapex_check_bash_safety_properties.py | 93 +++++++++++++++++- 3 files changed, 206 insertions(+), 17 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 4f90cfa0..076eca2a 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3524,17 +3524,16 @@ def _token_is_a_vanishing_default_or_alt_clause(token: str, name_to_raw_value: d def _redirect_span_length(seg: list[str], j: int) -> int: """The number of tokens, starting at SEG[j], that make up one bash - I/O-redirection clause -- an optional leading bare file-descriptor - number (`2>`), a redirect operator (`_REDIRECT_OPERATORS`), and - exactly one target token. `segment_tokens` never splits a segment at - `<`/`>` (see `_SINGLE_OPS`'s own docstring -- a redirect may legally - sit anywhere before a command word without being that word itself), - so each piece of the clause survives here as its own separate token - (confirmed via `tokenize()`: `git > /dev/null checkout` tokenizes to - `['git', '>', '/dev/null', 'checkout']`; `git 2> /dev/null checkout` - to `['git', '2', '>', '/dev/null', 'checkout']`). Returns 0 when - SEG[j:] does not start with this shape, so a caller can treat 0 as - "not a redirect, do not skip" without a separate boolean check. + I/O-redirection clause -- a redirect operator (`_REDIRECT_OPERATORS`) + and exactly one target token. `segment_tokens` never splits a segment + at `<`/`>` (see `_SINGLE_OPS`'s own docstring -- a redirect may + legally sit anywhere before a command word without being that word + itself), so each piece of the clause survives here as its own + separate token (confirmed via `tokenize()`: `git > /dev/null + checkout` tokenizes to `['git', '>', '/dev/null', 'checkout']`). + Returns 0 when SEG[j:] does not start with this shape, so a caller + can treat 0 as "not a redirect, do not skip" without a separate + boolean check. CRITICAL bug found by independent adversarial review (round 14, issue #1375) and independently reproduced live: every existing skip-past- @@ -3552,7 +3551,76 @@ def _redirect_span_length(seg: list[str], j: int) -> int: which is neither vanishing nor dynamic, so the real, cd-resolving `$X` one position later was never checked) -- both live-verified (real bash, and end-to-end through the real wrapper against a - scratch git repo) to silently discard a genuinely dirty file.""" + scratch git repo) to silently discard a genuinely dirty file. + + Deliberately does NOT also recognize an optional leading bare + file-descriptor number (`2>`) as part of the clause, despite that + being real bash syntax -- CRITICAL data-loss bug found by independent + adversarial review (round 16, issue #1375) and independently + reproduced live: this function's own first version did consume a + leading digit token, but `tokenize()`'s own shlex punctuation-based + splitting produces the IDENTICAL token sequence for `2>file` (a + fused, genuine fd-redirect prefix -- no argument) and `2 >file` (the + literal word `2` followed by a separate, ordinary stdout redirect -- + a real argument this classifier must not lose) -- the raw source's + own whitespace adjacency between the digit and the operator, the only + signal that actually distinguishes the two, is already gone by the + time this function ever sees the tokens. Confirmed live (a real-bash + argv-capture proxy) that the spaced form genuinely passes `2` through + as a real, separate argument. Consuming the digit unconditionally + silently dropped that argument from `checkout_restore_paths` -- + `git checkout -- realfile.py 2 >target.txt` (`2` a real, dirty, + tracked file) resolved to `checkout_restore_paths=('realfile.py',)`, + missing `2` entirely, and `git restore --source 2 >target.txt + file.py` resolved to an EMPTY `checkout_restore_paths` (worse: once + `2` vanished, `--source`'s own value-consumption swallowed `file.py` + itself, the actual restore target) -- both live-verified end-to-end + through the real wrapper to silently discard a genuinely dirty file. + Leaving a leading digit token OUT of the redirect span instead makes + it survive as an ordinary candidate word wherever this function's + callers use it as one (a path token, a possible command word) -- + over-inclusion in the rarer, genuinely-fused case, the safe direction + this module takes everywhere else, rather than the file-descriptor + heuristic's own proven under-inclusion.""" + if j < len(seg) and seg[j] in _REDIRECT_OPERATORS and j + 1 < len(seg): + return 2 + return 0 + + +def _redirect_span_length_with_optional_fd(seg: list[str], j: int) -> int: + """Like `_redirect_span_length`, but ALSO recognizes an optional + leading bare file-descriptor number (`2>`) as part of the clause -- + for callers trying to SKIP PAST a possible redirect to find something + else beyond it (`_find_git_checkout_restore`'s own global-flag-skip + loop, `_first_surviving_segment_word`'s own leading-decoy walk), never + for callers extracting `checkout_restore_paths` candidates + (`_strip_redirect_clauses`, which deliberately uses the strict, + digit-free `_redirect_span_length` instead -- see that function's own + docstring for why the two contexts need OPPOSITE defaults for the + same digit-adjacency ambiguity). + + `tokenize()`'s own shlex punctuation-splitting cannot distinguish a + fused `2>file` (a genuine fd-redirect prefix, no argument) from a + spaced `2 >file` (the literal word `2` followed by a separate + redirect) -- both produce the identical token sequence, so this + ambiguity is undecidable at the token level regardless of which + default a caller picks (see `_redirect_span_length`'s own docstring, + round-16 finding, for the live-verified data-loss risk of guessing + "consumed by the redirect" in a PATH-extraction context). Here, in a + skip-PAST context, the risk runs the other way: NOT skipping a + genuine fd-prefixed redirect (e.g. `2>&1`) makes the walk stop on the + bare digit token itself -- neither a flag, a vanishing decoy, nor + `checkout`/`restore` -- so a fully literal, unambiguous `git > out.log + 2>&1 checkout -- dirty.py` would go entirely unrecognized as a + checkout invocation at all, the FAIL-OPEN direction for this walk + (an empty `checkout_restore_paths` means the live wrapper check never + even runs) -- confirmed live as a regression the strict, digit-free + version introduces here specifically, during this same round's own + fix. Skipping the digit here, even when it was actually a real + argument in the spaced-form reading, costs nothing extra: the + literal-token scan these callers sit alongside already checks every + token in the segment regardless of position, so a real decoy or + relocator sitting at that position is not hidden by this skip.""" n = len(seg) i = j if i < n and seg[i].isdigit(): @@ -3743,7 +3811,7 @@ def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str] continue ambiguous = True break - redirect_len = _redirect_span_length(seg, j) + redirect_len = _redirect_span_length_with_optional_fd(seg, j) if redirect_len: j += redirect_len continue @@ -3889,7 +3957,7 @@ def _first_surviving_segment_word(seg: list[str], name_to_raw_value: dict[str, s ): i += 1 continue - redirect_len = _redirect_span_length(seg, i) + redirect_len = _redirect_span_length_with_optional_fd(seg, i) if redirect_len: i += redirect_len continue diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 23fc4b98..f1485d23 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1490,6 +1490,40 @@ def test_checkout_allowed_when_a_trailing_redirect_names_an_unrelated_dirty_file assert result.stderr == "" +def test_checkout_denied_when_a_digit_shaped_path_sits_before_a_redirect(tmp_path: Path) -> None: + """CRITICAL data-loss regression pin (round-16 independent review, + issue #1375). `tokenize()`'s own shlex punctuation-splitting cannot + distinguish a fused `2>file` (a genuine fd-redirect prefix, no + argument) from a spaced `2 >file` (the literal word `2` followed by + a separate redirect) -- both produce the identical token sequence. + The classifier's own former digit-consuming redirect heuristic + wrongly guessed "consumed by the redirect" here, silently dropping a + real, dirty, tracked file named `2` from `checkout_restore_paths`. + `realfile.py` here is clean; `2` is the only genuinely dirty file.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir, filename="realfile.py") + file_path = _init_repo_with_committed_file(repo_dir, filename="2") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("git checkout -- realfile.py 2 >target.txt", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_checkout_denied_behind_multiple_redirects_including_a_digit_prefixed_one(tmp_path: Path) -> None: + """CRITICAL false-negative regression pin (round-16 independent + review, issue #1375, own follow-up). Making the strict, digit-free + redirect check the ONLY one in use would make the subcommand-finding + walk stop on a bare digit token sitting in front of a genuine + `2>&1`-shaped redirect, so a fully literal, unambiguous checkout + behind multiple redirects would go entirely unrecognized (the live + wrapper check never even running) -- confirmed as a regression this + same round's own fix would otherwise introduce.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("git > out.log 2>&1 checkout -- dirty.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_for_a_real_checkout_hidden_behind_a_commented_paren_in_a_substitution( tmp_path: Path, ) -> None: diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index a7dad4bf..ac8addf8 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -3722,8 +3722,7 @@ def test_classify_denies_a_checkout_hidden_behind_a_leading_vanishing_decoy() -> def test_redirect_span_length_recognizes_operator_and_target() -> None: assert checker._redirect_span_length([">", "/dev/null", "x"], 0) == 2 - assert checker._redirect_span_length(["2", ">", "/dev/null", "x"], 0) == 3 - assert checker._redirect_span_length(["2", ">&", "1", "x"], 0) == 3 + assert checker._redirect_span_length([">&", "1", "x"], 0) == 2 def test_redirect_span_length_zero_when_no_redirect_present() -> None: @@ -3855,10 +3854,18 @@ def test_classify_does_not_extract_paths_from_an_unrelated_dynamic_tool() -> Non def test_strip_redirect_clauses_removes_a_redirect_wherever_it_occurs() -> None: assert checker._strip_redirect_clauses(["f.py", ">>", "log.txt"]) == ["f.py"] assert checker._strip_redirect_clauses([">>", "log.txt", "f.py"]) == ["f.py"] - assert checker._strip_redirect_clauses(["f.py", "2", ">&", "1"]) == ["f.py"] + assert checker._strip_redirect_clauses(["f.py", ">&", "1"]) == ["f.py"] assert checker._strip_redirect_clauses(["f.py", "g.py"]) == ["f.py", "g.py"] +def test_strip_redirect_clauses_preserves_a_leading_digit_as_a_real_token() -> None: + """CRITICAL data-loss regression pin (round-16 independent review, + issue #1375): the strict, path-extraction-facing variant must NOT + guess "consumed by the redirect" for a leading digit token -- see + `_redirect_span_length`'s own docstring for why.""" + assert checker._strip_redirect_clauses(["f.py", "2", ">", "target.txt"]) == ["f.py", "2"] + + def test_git_checkout_paths_excludes_a_trailing_redirect_clause() -> None: """CRITICAL false-positive regression pin (round-15 independent review, issue #1375). Round 14 taught `_find_git_checkout_restore` @@ -3895,6 +3902,86 @@ def test_classify_does_not_flag_a_redirect_target_as_a_checkout_path() -> None: assert verdict.checkout_restore_paths == ("f.py",), cmd +def test_redirect_span_length_never_consumes_a_leading_digit() -> None: + """CRITICAL data-loss regression pin (round-16 independent review, + issue #1375). `tokenize()`'s own shlex punctuation-splitting cannot + distinguish a fused `2>file` (a genuine fd-redirect prefix, no + argument) from a spaced `2 >file` (the literal word `2` followed by + a separate redirect) -- both produce the identical token sequence. + The strict, path-extraction-facing `_redirect_span_length` must NOT + guess "consumed by the redirect" for the leading digit: doing so + silently drops a real argument from `checkout_restore_paths`.""" + assert checker._redirect_span_length(["2", ">", "target.txt"], 0) == 0 + assert checker._redirect_span_length([">", "target.txt"], 0) == 2 + + +def test_git_checkout_paths_does_not_drop_a_digit_shaped_path() -> None: + """CRITICAL data-loss regression pin (round-16 independent review, + issue #1375). Live-verified before this fix: `git checkout -- + realfile.py 2 >target.txt` resolved to + `checkout_restore_paths=('realfile.py',)`, silently dropping `2` (a + real, dirty, tracked file) -- the classifier's own former digit- + consuming redirect heuristic wrongly treated `2` as an fd-redirect + prefix rather than a real path argument.""" + deny_reason, paths = checker._git_checkout_paths(["--", "realfile.py", "2", ">", "target.txt"], {}) + assert deny_reason is None + assert paths == ("realfile.py", "2") + + +def test_git_restore_paths_does_not_drop_a_real_path_behind_a_digit_redirect() -> None: + """CRITICAL data-loss regression pin (round-16 independent review, + issue #1375). Live-verified before this fix: `git restore --source 2 + >target.txt file.py` resolved to an EMPTY `checkout_restore_paths` -- + once `2` vanished into the wrongly-recognized redirect, `--source`'s + own value-consumption swallowed `file.py` itself, the actual restore + target, leaving nothing for the live wrapper check to examine.""" + deny_reason, paths = checker._git_restore_paths(["--source", "2", ">", "target.txt", "file.py"], {}) + assert deny_reason is None + assert paths == ("file.py",) + + +def test_classify_does_not_drop_a_digit_shaped_path_behind_a_redirect() -> None: + """End-to-end regression pin for the round-16 finding at the + `classify()` level.""" + verdict = checker.classify("git checkout -- realfile.py 2 >target.txt") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("realfile.py", "2") + + +def test_redirect_span_length_with_optional_fd_recognizes_a_fused_fd_redirect() -> None: + """CRITICAL false-negative regression pin (round-16 independent + review, issue #1375, own follow-up). The strict, digit-free + `_redirect_span_length` alone made the subcommand-finding/cwd- + relocation walks stop on a bare digit token sitting in front of a + genuine `2>&1`-shaped redirect, so a fully literal, unambiguous `git + > out.log 2>&1 checkout -- dirty.py` went entirely unrecognized as a + checkout invocation (empty `checkout_restore_paths`, the live + wrapper check never runs at all) -- the FAIL-OPEN direction for this + walk. `_redirect_span_length_with_optional_fd` (used only by the + skip-PAST-a-possible-redirect walks, never by path extraction) + closes this by also consuming an optional leading digit.""" + assert checker._redirect_span_length_with_optional_fd(["2", ">&", "1", "checkout"], 0) == 3 + assert checker._redirect_span_length_with_optional_fd([">", "out.log", "checkout"], 0) == 2 + assert checker._redirect_span_length_with_optional_fd(["checkout"], 0) == 0 + + +def test_find_git_checkout_restore_skips_a_digit_prefixed_redirect_between_git_and_subcommand() -> None: + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore( + ["git", ">", "out.log", "2", ">&", "1", "checkout", "--", "f.py"], {} + ) + assert subcommand == "checkout" + assert tokens_after == ["--", "f.py"] + assert saw_tree_relocation is False + + +def test_classify_extracts_paths_behind_multiple_redirects_including_a_digit_prefixed_one() -> None: + """End-to-end regression pin for the round-16 follow-up finding at + the `classify()` level.""" + verdict = checker.classify("git > out.log 2>&1 checkout -- dirty.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + # --- End-to-end classify() coverage, pinning every explicit safe/deny case # issue #1375's own Acceptance Criteria Map and "Explicit safe cases" # section name by hand. From 656f6850456e7be5d9d06b6966f0a71440743141 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 16:08:05 +0000 Subject: [PATCH 22/46] docs(hooks): disclose two round-17 redirect-handling residuals A fresh, independent adversarial review of this PR's current head (round 17) found two issues in the redirect-clause handling added across rounds 14-16, both rooted in the same underlying limitation: tokenize()'s own shlex dequotes every token before _REDIRECT_OPERATORS membership is ever checked, so a token's final text alone cannot tell a genuine, unquoted redirect operator apart from a quoted literal that happens to dequote to the same text. Finding 1 (CRITICAL, disclosed via a new dedicated issue, not fixed in-PR): a real, tracked file literally named one of the nine _REDIRECT_OPERATORS strings (e.g. ">") defeats path extraction entirely. Live-verified: with a real tracked file named ">" and a second, genuinely dirty file realfile.py, git checkout ">" realfile.py genuinely discards realfile.py's uncommitted content when actually executed, while classify() reports an EMPTY checkout_restore_paths -- _strip_redirect_clauses misreads the quoted ">" as a real operator and realfile.py as its "target," stripping both. A narrow fix confined to the redirect-handling functions alone does not exist without reintroducing round 15's own, far more common false positive (denying an ordinary git checkout -- f.py >> log.txt-style output redirect); a genuine fix needs tokenize() itself to preserve per-token quote/escape provenance, the same class of tokenizer-level change issue #1404 already requires. Tracked as issue #1412, disclosed in the module's own header docstring (matching the #1404 precedent immediately above it) and pinned as quoted-redirect-operator-shaped-filename-bypass in KNOWN_BYPASS_COMMANDS. Finding 2 (accepted trade-off, not a defect): round 16's own deliberate choice to leave a leading digit token out of a redirect span (rather than risk dropping a real path argument) is inherently indistinguishable, at the token level, from the case where the digit genuinely is consumed by a fused N>file-shaped redirect with no -- present -- both tokenize identically. This can flip _git_checkout_paths's own sub-case (b) count-based decision and extract an extra, harmless-to-check candidate path, producing a false deny when that path happens to be dirty even though the fused reading never actually touches it. This is the same "when a token-level ambiguity cannot be soundly resolved, prefer extra scrutiny over a silent miss" posture this module already takes everywhere else (e.g. round 12's own still-dynamic-candidate fail-closed choice) -- an accepted, intentional cost of round 16's own data-loss fix, documented inline at both the module docstring and _git_checkout_paths's own sub-case (b). No code behavior changes in this commit -- documentation and a regression pin only. Refs #1375, #1412. --- hooks/gitapex_check_bash_safety.py | 68 ++++++++++++++++++++++++- hooks/test_gitapex_check_bash_safety.py | 29 +++++++++++ 2 files changed, 96 insertions(+), 1 deletion(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 076eca2a..5355fc20 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -105,6 +105,64 @@ inside-command-substitution-full-bypass` in hooks/test_gitapex_check_bash_safety.py's own `KNOWN_BYPASS_COMMANDS`. +CRITICAL, disclosed, redirect-handling-specific limitation, the same +underlying `shlex`-quote-information-loss class as the residual just +above (found live by Step 8 independent review, round 17 of issue +#1375's own checkout/restore feature review, while stress-testing +rounds 14-16's own newly-added redirect-clause handling): +`_REDIRECT_OPERATORS`/`_redirect_span_length`/`_redirect_span_length_ +with_optional_fd`/`_strip_redirect_clauses` recognize a redirect +operator purely by a token's final TEXT value -- `tokenize()`'s own +`shlex` dequotes every token before this check ever runs, so a real, +tracked file whose name is literally one of these operator strings +(e.g. a file named `>`) tokenizes to the exact same text as a genuine, +unquoted redirect operator, with no way to recover which one the +source actually was. Live-verified real, silent data loss: with a +real tracked file literally named `>` and a second file `realfile.py` +genuinely dirty, `git checkout ">" realfile.py` genuinely discards +`realfile.py`'s uncommitted content when actually executed, while +`classify()` reports `deny=False` with an EMPTY `checkout_restore_ +paths` -- `_strip_redirect_clauses` misreads the quoted `">"` as a +real operator and `realfile.py` as its "target," stripping BOTH and +leaving nothing for `_git_checkout_paths` to extract. Reachability is +narrow (the decoy path must already exist as a real tracked file, or +git's own atomic multi-pathspec validation aborts first, confirmed +live), but the underlying gap is real and this is a full, confirmed +bypass for that shape. Deliberately NOT attempted here: a narrow fix +confined to the redirect-handling functions alone does not exist +without reintroducing round 15's own, far more common false-positive +(denying an ordinary `git checkout -- f.py >> log.txt`-style output +redirect) -- a genuine fix needs `tokenize()` itself to preserve +per-token quote/escape provenance, the same class of tokenizer-level +change issue #1404 above already requires, not a narrow patch; tracked +as its own dedicated issue, https://github.com/tvna/gitapex/issues/1412, +since it is a distinct shlex-information-loss shape (redirect-operator +text matching, not nested double-quote state) from #1404's own finding; +pinned as `quoted-redirect-operator-shaped-filename-bypass` in +hooks/test_gitapex_check_bash_safety.py's own `KNOWN_BYPASS_COMMANDS`. + +A second, distinct, round-17 finding in the SAME redirect-handling area +is NOT a bypass and is NOT tracked separately: `_redirect_span_length`'s +own deliberate choice (round 16) to leave a leading digit token OUT of +a redirect span, rather than risk dropping a real path argument (see +that function's own docstring), is INHERENTLY indistinguishable at the +token level from the case where the digit genuinely is consumed by a +fused `N>file`-shaped redirect with no `--` present -- `git checkout +realfile.py 2>target.txt` (fused, no `--`) and `git checkout realfile.py +2 >target.txt` (spaced, no `--`) tokenize identically, but only the +spaced form genuinely passes `2` to git as a second positional. Since +this classifier cannot tell the two apart, it deliberately treats BOTH +the same way -- as `_git_checkout_paths`'s own sub-case (b), extracting +BOTH `realfile.py` and `2` -- rather than silently assuming the +fused-and-therefore-safe reading. This can produce a false deny when a +file named `2` happens to be dirty even though the fused form never +actually touches it, but that is the SAME "when a token-level ambiguity +cannot be soundly resolved, prefer extra scrutiny over a silent miss" +posture this module takes everywhere else (see round 12's own still- +dynamic-candidate fail-closed choice for the identical trade-off +direction) -- an accepted, intentional cost of round 16's own data-loss +fix, not a new defect. + Closed by fifth-round Step 8 independent review: `_gh_api_method_dynamic_ value`/`_gh_api_field_dynamic_hit` (and the earlier literal-token scans) only ever recognized a dynamic VALUE fused onto a literal `-X`/`--method`/ @@ -3234,7 +3292,15 @@ def _git_checkout_paths( `-b`/`-B`/`--orphan` (see below), every position past the first is a pathspec under every resolution git can take. Over-including a token that also happens to be a valid ref name just checks a path - that likely does not exist, which is harmless. + that likely does not exist, which is harmless. A digit token that + survived `_strip_redirect_clauses`'s own deliberately strict, + never-consume-a-leading-digit choice (see `_redirect_span_length`'s + own docstring, round-16 finding) can flip a genuinely single- + positional, no-`--` invocation (case Non-goal below) into THIS + case when the digit was actually consumed by a real, fused + `N>file`-shaped redirect at real bash runtime -- an accepted, + intentional cost of that fix (see the module docstring's own + round-17 paragraph for the full trade-off), not a defect. (c) No `--`, exactly one positional token, and it is the literal `.` or `..` -- both are syntactically invalid git ref names (confirmed live: `git check-ref-format --branch .`/`--branch ..` both fail, diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index f1485d23..021842ba 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -448,6 +448,35 @@ def assert_allowed(command: str) -> None: 'x="$(echo "y)" && git checkout -- dirty.py)"', "shlex-nested-double-quote-inside-command-substitution-full-bypass", ), + ( + # CRITICAL bypass, same underlying shlex-quote-information-loss + # class as the residual just above. Found live by independent + # adversarial review (round 17, issue #1375), tracked as its own + # dedicated issue rather than fixed here: + # https://github.com/tvna/gitapex/issues/1412 -- deliberately out + # of issue #1375's own scope, since a narrow fix confined to the + # redirect-handling functions alone does not exist without + # reintroducing round 15's own, far more common false positive + # (denying an ordinary `git checkout -- f.py >> log.txt`-style + # output redirect); a genuine fix needs tokenize() itself to + # preserve per-token quote/escape provenance, the same class of + # tokenizer-level change issue #1404 already requires. + # `_REDIRECT_OPERATORS`/`_strip_redirect_clauses` recognize a + # redirect operator purely by a token's final TEXT value -- + # `tokenize()`'s own shlex dequotes every token first, so a real, + # tracked file literally named `>` tokenizes identically to a + # genuine, unquoted redirect operator. Live-verified real, silent + # data loss: with a real tracked file literally named `>` and a + # second, genuinely dirty file `realfile.py`, this exact command + # discards `realfile.py`'s uncommitted content when actually + # executed, while `classify()` reports `deny=False` with an EMPTY + # `checkout_restore_paths` -- the quoted `">"` is misread as a + # real operator and `realfile.py` as its "target," stripping both + # and leaving nothing to extract. See issue #1412 for the full + # write-up and live-verification detail. + 'git checkout ">" realfile.py', + "quoted-redirect-operator-shaped-filename-bypass", + ), ] From edc316864a17975eec60a383e167b66e03d069af Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 16:32:00 +0000 Subject: [PATCH 23/46] fix(hooks): thread outer scope into command-substitution recursion A fresh, independent adversarial review of this PR's current head (round 18) found that `_rule_command_substitution_content`'s own recursive `_classify_tokens(inner_tokens)`/`classify(inner_text)` calls passed no outer shell scope at all, so a `git` token itself held in a variable assigned OUTSIDE a `$(...)` command substitution -- an ordinary "hold the tool name in a variable" idiom, not an exotic precondition -- defeated checkout/restore recognition ENTIRELY, not merely under-extracted it. Since "checkout"/"restore" are not in `_WATCHED_VERBS`, there is no B1a/B1b fallback the way there might be for a `pip install`-shaped denial: an unresolvable `$G` left the substitution's own inner content invisible to `_find_git_checkout_ restore`'s own git-token recognition entirely, producing a silently EMPTY `checkout_restore_paths` and `deny=False` -- the exact failure mode issue #1375 exists to prevent. Live-verified before this fix: `G=git; x=$($G checkout -- dirty.py)` was wrongly allowed outright through the real wrapper, and the real command afterward silently discarded a genuinely dirty, tracked `dirty.py`. This same outer-scope gap was already disclosed as a residual in `_rule_array_literal_content`'s own nineteenth-round docstring paragraph (issue #1326), deferred at the time as a larger change than that round's own narrower finding (generic tool/verb reconstruction, e.g. `pip install`) warranted -- closing it needed `classify()`'s own public, string-based entry point (used for the quoted/fused `$(...)` shape) to also accept an outer scope. This round's own finding shows the same gap has a materially more severe consequence specifically for checkout/restore -- a full, silent bypass with real data loss, not merely an under-extraction -- so the larger change is made here. Fixed by: - Extending `classify()`'s signature with two new optional `outer_name_to_value`/`outer_name_to_raw_value` parameters (default `None`, preserving every existing 1-arg caller's behavior exactly), threaded straight through to `_classify_tokens`. - Extending `_rule_command_substitution_content`'s signature to require `name_to_value`/`name_to_raw_value` (mirroring `_rule_array_ literal_content`'s own parameters exactly), threaded into both the unquoted `_classify_tokens(inner_tokens, ...)` and the quoted/fused `classify(inner_text, ...)` recursive calls. - `_classify_tokens` now computes the outer-merged scope once and passes the same dicts to both `_rule_command_substitution_content` and `_rule_array_literal_content`, removing the previous duplicate computation for the array-literal call alone. Both call shapes are covered end to end (`hooks/test_gitapex_check_ bash_safety.py`, through the real wrapper against a scratch git repo with a genuinely dirty tracked file) and at the unit/property level (`tests/test_gitapex_check_bash_safety_properties.py`, mirroring the nineteenth round's own array-literal outer-scope tests). Full gate suite green: ruff check/format, mypy, xenon (CI thresholds), the detection-logic property-coverage gate (no new trigger call sites), and 100% line+branch coverage on hooks/gitapex_check_bash_safety.py via the combined properties/end-to-end/oracle-pins/differential suite. A separate, narrower double-indirection gap in the underlying variable resolution itself (`G="$A$B"`, a raw value that is itself another unresolved reference rather than a plain literal) was found live during this round's own verification and confirmed to be a pre-existing, architecture-wide limitation of `_substitute_var_refs_candidates`'s single-pass substitution -- present identically at the top level with no command substitution involved at all, predating issue #1375 by a wide margin, and already covered by `_dynamic_token_resolves_only_to_ literal`'s own documented fail-closed posture on an unresolvable candidate. Not a regression from this commit and not disclosed separately here. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 126 ++++++++++++------ hooks/test_gitapex_check_bash_safety.py | 35 +++++ ...st_gitapex_check_bash_safety_properties.py | 61 ++++++++- 3 files changed, 177 insertions(+), 45 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 5355fc20..29232e39 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -846,7 +846,9 @@ def _is_unresolvable_substitution(token: str) -> bool: return "$(" in token or "`" in token -def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, bool, tuple[str, ...]]: +def _rule_command_substitution_content( + tokens: list[str], name_to_value: dict[str, str], name_to_raw_value: dict[str, str] +) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `$(...)` command-substitution span's OWN inner content through this module's full rule set -- bash genuinely RUNS that inner text as a complete command the instant the @@ -910,18 +912,44 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b `is_git_push` bug would have dropped it -- the identical bug class, for a tuple instead of a bool. - Disclosed residual (found live by Step 8 independent review, - nineteenth round, issue #1326): unlike `_rule_array_literal_content`'s - own nineteenth-round fix, the recursive `_classify_tokens(inner_ - tokens)`/`classify(inner_text)` calls below pass no outer scope, so a - tool/verb built from a variable assigned OUTSIDE a `$(...)` span's own - text (e.g. `T=pip; V=install; x=$($T $V foo)`) is still invisible to - this recursive check, even though it resolves to a real denied - invocation at bash runtime. Not fixed here: closing it needs the - string-based `classify()` entry point (used for the quoted/fused - `$(...)` shape) to also accept an outer scope, a larger change than - the finding that prompted `_rule_array_literal_content`'s own fix - warranted.""" + NAME_TO_VALUE/NAME_TO_RAW_VALUE are the OUTER command's own assigned + variables, already merged with this span's own containing token + list's assignments by `_classify_tokens` before this function is ever + called -- mirrors `_rule_array_literal_content`'s own NAME_TO_VALUE/ + NAME_TO_RAW_VALUE parameters exactly (see that function's own + docstring), since a bare `$G` inside a `$(...)` span genuinely + resolves against the SAME shell scope as the rest of the command at + real bash runtime, not just against whatever the substitution's own + inner tokens happen to assign. + + CRITICAL bug found live by Step 8 independent review, eighteenth + round (issue #1375): an earlier version of this function passed no + scope at all to either recursive call, re-deriving nothing from the + substitution's own inner tokens either -- so a `git` token itself + held in a variable assigned OUTSIDE the `$(...)` span (e.g. `G=git; + x=$($G checkout -- dirty.py)`, an ordinary "hold the tool name in a + variable" idiom, not an exotic precondition) defeated checkout/ + restore recognition ENTIRELY, not merely under-extracted it: since + "checkout"/"restore" are not in `_WATCHED_VERBS`, there is no B1a/B1b + fallback the way there might be for a `pip install`-shaped denial, + so an unresolvable `$G` left the substitution's own inner content + invisible to `_find_git_checkout_restore`'s own git-token recognition + entirely, producing a silently EMPTY `checkout_restore_paths` and + `deny=False` -- the exact failure mode issue #1375 exists to prevent. + Confirmed live end-to-end through the real wrapper against a scratch + git repo with a genuinely dirty, tracked `dirty.py`: the wrapper + returned exit 0 (allow) for `G=git; x=$($G checkout -- dirty.py)`, + and actually running that command afterward reverted `dirty.py` to + its committed content, discarding the uncommitted edit. Closed by + threading NAME_TO_VALUE/NAME_TO_RAW_VALUE into both the unquoted + `_classify_tokens(inner_tokens, ...)` and the quoted/fused + `classify(inner_text, ...)` recursive calls below -- the latter + required extending `classify()`'s own public, string-based entry + point to accept the same optional outer-scope pair + `_classify_tokens` already did -- the exact extension + `_rule_array_literal_content`'s own nineteenth-round paragraph (see + that function's docstring) once deferred as a larger change than its + own, narrower finding warranted.""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -949,7 +977,7 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b # check would only change how often an empty-content recursive # `classify()` call is skipped, never a real verdict. if inner_text.strip(): - inner_verdict = classify(inner_text) + inner_verdict = classify(inner_text, name_to_value, name_to_raw_value) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) if inner_verdict.deny: @@ -963,7 +991,7 @@ def _rule_command_substitution_content(tokens: list[str]) -> tuple[str | None, b if span_end is not None: inner_tokens = tokens[i + 2 : span_end - 1] if inner_tokens: - inner_verdict = _classify_tokens(inner_tokens) + inner_verdict = _classify_tokens(inner_tokens, name_to_value, name_to_raw_value) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) if inner_verdict.deny: @@ -2301,14 +2329,17 @@ def _rule_array_literal_content( (confirmed live via `declare -p`) the SAME way they would if `$G $P $M` appeared directly at the top level of the command instead of inside an array literal. Closed by threading the outer scope through. - Disclosed residual: `_rule_command_substitution_content`'s own, - pre-existing (since the fourteenth round) recursive checks have the - identical outer-scope gap and are NOT fixed by this round -- a tool/ - verb built from a variable assigned outside a `$(...)` span's own - text is still invisible to that recursive check. Not closed here: - closing it needs `classify()`'s own string-based entry point (used - for the quoted/fused `$(...)` shape) to also accept an outer scope, - a larger change than this round's own confirmed finding warranted. + At the time of this round, `_rule_command_substitution_content`'s own, + pre-existing (since the fourteenth round) recursive checks had the + identical outer-scope gap and were left as a disclosed residual, since + closing it needed `classify()`'s own string-based entry point (used + for the quoted/fused `$(...)` shape) to also accept an outer scope -- + a larger change than this round's own confirmed finding warranted. A + later round (eighteenth, issue #1375) found that gap defeated + checkout/restore recognition entirely, not merely under-extracted it, + and closed it the same way this round closed the array-literal + equivalent -- see `_rule_command_substitution_content`'s own + docstring for that finding and fix. Found live during independent adversarial review of issue #1350's own newline fix: `NAME=(...)` parens denote a bash WORD LIST (a compound @@ -4341,16 +4372,30 @@ def _segment_loop_hit( return None, is_git_push -def classify(command: str) -> Verdict: +def classify( + command: str, + outer_name_to_value: dict[str, str] | None = None, + outer_name_to_raw_value: dict[str, str] | None = None, +) -> Verdict: """Classify one Bash tool_input.command string. Fails closed (deny) on anything shlex cannot tokenize -- an unparseable command is exactly the "cannot confidently classify" case dimension 15 requires denying, not - silently allowing.""" + silently allowing. + + OUTER_NAME_TO_VALUE/OUTER_NAME_TO_RAW_VALUE, when given, are passed + straight through to `_classify_tokens` -- see that function's own + docstring for what they mean and who supplies them. Every ordinary + caller (the module's own entrypoint, tests) omits them and gets this + function's original, scope-free behavior exactly; `_rule_command_ + substitution_content`'s own recursive call (issue #1375, eighteenth + round) is the one caller that supplies them, so a quoted/fused + `$(...)` span's own inner content can resolve a variable assigned + OUTSIDE the span against the same shell scope real bash would use.""" try: tokens = tokenize(command) except TokenizeError as error: return Verdict(True, f"the command could not be parsed as shell syntax ({error}). Failing closed", False) - return _classify_tokens(tokens) + return _classify_tokens(tokens, outer_name_to_value, outer_name_to_raw_value) def _classify_tokens( @@ -4376,26 +4421,29 @@ def _classify_tokens( inner-scope reassignment does shadow the outer one). Named to match every other function in this module that takes this same pair (`name_to_value`/`name_to_raw_value`), not a new vocabulary of their - own. Used only by `_rule_array_literal_content`'s own recursive call, - to give an array literal's own inner content access to the SAME - shell scope as the rest of the command (see that function's own - docstring, nineteenth-round paragraph, for the live bypass this - closes) -- `None` (every other caller, including the top-level - `classify` and `_rule_command_substitution_content`'s own recursive - calls -- see that function's own docstring for the disclosed residual - this leaves there) preserves this function's prior, scope-free - behavior exactly.""" + own. Threaded into BOTH `_rule_array_literal_content`'s and + `_rule_command_substitution_content`'s own recursive calls, so an + array literal's or a `$(...)` span's own inner content each get + access to the SAME shell scope as the rest of the command (see each + function's own docstring -- the nineteenth-round paragraph for the + array-literal fix, the eighteenth-round paragraph for the command- + substitution fix -- for the live bypass each closes) -- `None` (the + top-level `classify` entry point, when its own caller has no outer + scope of its own to supply) preserves this function's behavior for a + genuinely top-level command exactly.""" outer_literals = outer_name_to_value or {} outer_raw = outer_name_to_raw_value or {} + merged_name_to_value = {**outer_literals, **_assigned_literals(tokens)} + merged_name_to_raw_value = {**outer_raw, **_assigned_raw_values(tokens)} - content_reason, content_is_git_push, content_checkout_restore_paths = _rule_command_substitution_content(tokens) + content_reason, content_is_git_push, content_checkout_restore_paths = _rule_command_substitution_content( + tokens, merged_name_to_value, merged_name_to_raw_value + ) if content_reason: return Verdict(True, content_reason, content_is_git_push, content_checkout_restore_paths) array_content_reason, array_content_is_git_push, array_content_checkout_restore_paths = _rule_array_literal_content( - tokens, - {**outer_literals, **_assigned_literals(tokens)}, - {**outer_raw, **_assigned_raw_values(tokens)}, + tokens, merged_name_to_value, merged_name_to_raw_value ) is_git_push = content_is_git_push or array_content_is_git_push checkout_restore_paths = content_checkout_restore_paths + array_content_checkout_restore_paths diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 021842ba..a2612be9 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1877,6 +1877,41 @@ def test_checkout_denied_when_git_itself_is_a_dynamic_word(tmp_path: Path) -> No assert result.returncode == 2, f"stderr={result.stderr!r}" +def test_checkout_denied_when_a_dynamic_git_token_sits_inside_a_command_substitution(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-18 independent review, issue + #1375). A `git` token held in a variable assigned OUTSIDE a `$(...)` + command substitution -- an ordinary "hold the tool name in a + variable" idiom, not an exotic precondition -- defeated checkout/ + restore recognition entirely, not merely under-extracted it: the + recursive classification of a substitution's own inner content + passed no outer scope, so `$G` could never resolve to `git` there + even though it unambiguously does at real bash runtime. Live-verified + before this fix: `G=git; x=$($G checkout -- dirty.py)` was wrongly + allowed outright through the real wrapper, and the real command + afterward silently discarded a genuinely dirty `dirty.py`.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("G=git; x=$($G checkout -- dirty.py)", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_checkout_denied_when_a_dynamic_git_token_sits_inside_a_quoted_command_substitution( + tmp_path: Path, +) -> None: + """Companion to the unquoted-form pin above, for the quoted/fused + `$(...)` shape (`_find_fused_command_substitution`, recursed into via + `classify()` on the inner TEXT rather than `_classify_tokens` on + inner TOKENS) -- the two shapes are handled by separate code paths in + `_rule_command_substitution_content`, both needed the outer-scope fix + (round-18 independent review, issue #1375).""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run('G=git; x="$($G checkout -- dirty.py)"', payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index ac8addf8..0467ac9f 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -1358,7 +1358,7 @@ def test_rule_command_substitution_content_detects_an_embedded_install(tool: str a punctuation character shlex breaks a word at, so an assignment's `NAME=` prefix stays fused onto the leading `$` in the same token.""" tokens = ["x=$", "(", tool, "install", "evil-pkg", ")"] - reason, _, _ = checker._rule_command_substitution_content(tokens) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}) assert reason is not None @@ -1374,7 +1374,7 @@ def test_rule_command_substitution_content_allows_harmless_inner_content(value: silently dropping a non-denying inner `is_git_push=True` signal (see the function's own docstring).""" tokens = ["echo", "$", "(", "date", value, ")"] - assert checker._rule_command_substitution_content(tokens) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}) == (None, False, ()) # --- Issue #1326 Stage 1, fifteenth round: bash's own leading-assignment ---- @@ -1670,6 +1670,55 @@ def test_classify_denies_array_literal_content_with_braced_decoy_end_to_end() -> assert verdict.deny is True +def test_rule_command_substitution_content_detects_an_outer_scope_resolved_checkout() -> None: + """Model-based, regression pin for the real bypass found live by Step + 8 independent review, eighteenth round (issue #1375): a `git` token + built from a variable assigned OUTSIDE the `$(...)` span's own + text (NAME_TO_VALUE's own entries, not anything the substitution's + own inner tokens assign) must still be recognized -- `G=git; x=$($G + checkout -- dirty.py)` was wrongly ALLOWED, with an EMPTY + `checkout_restore_paths`, before outer scope was threaded into the + recursive `_classify_tokens` call below. Mirrors `_rule_array_ + literal_content`'s own nineteenth-round test of the identical shape + for the array-literal span.""" + tokens = ["x=$", "(", "$G", "checkout", "--", "dirty.py", ")"] + outer = {"G": "git"} + reason, _, checkout_restore_paths = checker._rule_command_substitution_content(tokens, outer, outer) + assert reason is None + assert checkout_restore_paths == ("dirty.py",) + + +def test_classify_extracts_command_substitution_checkout_paths_with_outer_scope_end_to_end() -> None: + """`classify()`'s own `checkout_restore_paths` extraction, reached + end-to-end -- not just the recursive rule's own unit test above. + Regression pin for the real bypass found live by Step 8 independent + review, eighteenth round (issue #1375): before the outer-scope fix, + this resolved to an EMPTY `checkout_restore_paths`, the same silent + "nothing to see here" this classifier's own `deny=False` gives every + ordinary checkout/restore invocation -- `deny` itself stays False + here regardless (this module never unconditionally denies checkout/ + restore; the live wrapper's own `git diff --quiet` check is what + turns a non-empty `checkout_restore_paths` into an actual deny, see + `hooks/check-bash-safety.sh`). Exercises the unquoted, cross-token + `$(...)` shape (`_command_substitution_token_span`, recursed into via + `_classify_tokens` on inner TOKENS).""" + verdict = checker.classify("G=git; x=$($G checkout -- dirty.py)") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + +def test_classify_extracts_quoted_command_substitution_checkout_paths_with_outer_scope_end_to_end() -> None: + """Companion to the end-to-end pin above, for the quoted/fused + `$(...)` shape (`_find_fused_command_substitution`, recursed into via + `classify()` on the inner TEXT rather than `_classify_tokens` on + inner TOKENS) -- the two shapes are separate code paths in `_rule_ + command_substitution_content`, both needed the outer-scope fix + (eighteenth round, issue #1375).""" + verdict = checker.classify('G=git; x="$($G checkout -- dirty.py)"') + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + @_PROPERTIES @given(name=_IDENTIFIERS, subscript=st.sampled_from(["0", "1", "@", "*", "$i"])) def test_token_is_all_unassigned_refs_recognizes_a_braced_subscript(name: str, subscript: str) -> None: @@ -1888,7 +1937,7 @@ def test_rule_command_substitution_content_scans_second_fused_span_in_same_token this test only proves that fix reached end-to-end through `_rule_command_substitution_content`'s own scan loop.""" tokens = ["echo", "$(echo ok)$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}) assert reason is not None @@ -1897,20 +1946,20 @@ def test_rule_command_substitution_content_skips_blank_fused_span_then_finds_den skipped without denying by itself, but scanning continues to the next fused span in the same token.""" tokens = ["echo", "$( )$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}) assert reason is not None def test_rule_command_substitution_content_both_fused_spans_harmless() -> None: tokens = ["echo", "$(echo ok)$(echo also-ok)"] - assert checker._rule_command_substitution_content(tokens) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}) == (None, False, ()) def test_rule_command_substitution_content_empty_unquoted_span_skipped() -> None: """An empty, unquoted `$()` substitution has no inner tokens to recurse into -- distinct from the fused/quoted empty-span case above.""" tokens = ["$", "(", ")"] - assert checker._rule_command_substitution_content(tokens) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}) == (None, False, ()) def test_tokenize_raises_on_unbalanced_quote() -> None: From d9e3fd89285f5df782891100a4d333dca5799987 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 17:07:07 +0000 Subject: [PATCH 24/46] fix(hooks): recognize a reassigned-after-use dynamic git token A fresh, independent adversarial review of this PR's current head (round 19) found that `_assigned_raw_values`'s own order-blind, last-occurrence-in-token-order-wins collapse meant an entirely ordinary shell idiom -- reusing a variable name for a later, unrelated purpose after it was already used as `git` -- silently defeated checkout/ restore recognition entirely: `TOOL=git; $TOOL checkout -- dirty.py; TOOL=npm` resolved `$TOOL`'s own dict entry to `"npm"` (the LAST assignment in token order), even though `$TOOL` genuinely was `git` at the actual point of use one statement earlier. Since "checkout"/ "restore" are not in `_WATCHED_VERBS`, there is no fallback coverage the way there might be for a `pip install`-shaped denial: the whole invocation went unrecognized, producing a silently EMPTY `checkout_restore_paths` and `deny=False`. Live-verified before this fix: the real wrapper allowed the command above outright (exit 0), and the real command afterward silently discarded a genuinely dirty, tracked `dirty.py`. Identically reproducible for `git restore`, and for the command-substitution shape round 18 just fixed (`G=git; x=$($G checkout -- dirty.py); G=notgit`). This is not itself a new gap -- `_assigned_raw_values`'s own order-blind collapse is a pre-existing, whole-module primitive, and the module already discloses the identical class of gap for `$IFS` reassignment elsewhere as an accepted limitation -- but checkout/ restore is the one consumer where a silent miss means real, irreversible data loss rather than a missed advisory warning, so a narrow, targeted mitigation is applied here rather than left disclosed-only. Fixed by adding `_assigned_raw_values_biased_toward(tokens, literal)`: like `_assigned_raw_values`, but once a name is assigned LITERAL at any point among tokens, that name stays LITERAL regardless of a later, different reassignment -- a bounded, one-directional bias toward the single safe-to-over-recognize reading, not full execution-order tracking (a far larger, Stage-2-class change this module's own header docstring already scopes out of Stage 1's static analysis). Threaded through as a new, parallel `..._git_biased` argument alongside every existing outer-scope parameter this module already carries for checkout/restore purposes: `classify()`, `_classify_tokens`, `_rule_command_substitution_content`, `_rule_array_literal_content`, `_rule_git_checkout_restore`, and `_find_git_checkout_restore` (whose outer git-token-recognition now tries the ordinary reading first, falling back to the git-biased reading only when the ordinary one declines) -- mirroring round 18's own outer-scope-threading pattern exactly, so the fix also correctly covers a git token reassigned after use INSIDE a command substitution or array literal, not only at the top level. Only ever widens recognition (an unrelated tool's own dynamic `checkout`/`restore`-shaped subcommand at worst triggers a spurious, reversible live `git diff` check), never narrows it -- every case the ordinary reading already resolved is unaffected. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 194 ++++++++++++++++-- hooks/test_gitapex_check_bash_safety.py | 44 ++++ ...st_gitapex_check_bash_safety_properties.py | 180 ++++++++++++---- 3 files changed, 358 insertions(+), 60 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 29232e39..fce43ce2 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -847,7 +847,10 @@ def _is_unresolvable_substitution(token: str) -> bool: def _rule_command_substitution_content( - tokens: list[str], name_to_value: dict[str, str], name_to_raw_value: dict[str, str] + tokens: list[str], + name_to_value: dict[str, str], + name_to_raw_value: dict[str, str], + name_to_raw_value_git_biased: dict[str, str], ) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `$(...)` command-substitution span's OWN inner content through this module's full rule set -- bash genuinely @@ -949,7 +952,15 @@ def _rule_command_substitution_content( `_classify_tokens` already did -- the exact extension `_rule_array_literal_content`'s own nineteenth-round paragraph (see that function's docstring) once deferred as a larger change than its - own, narrower finding warranted.""" + own, narrower finding warranted. + + NAME_TO_RAW_VALUE_GIT_BIASED (round 19, issue #1375) is threaded + through the same two recursive calls alongside NAME_TO_VALUE/NAME_TO_ + RAW_VALUE, so a `git` token held in a variable that is reassigned + elsewhere in the OUTER command -- e.g. `G=git; x=$($G checkout -- + dirty.py); G=notgit` -- is still recognized inside the substitution's + own inner content; see `_find_git_checkout_restore`'s own docstring + for what this parameter means and the live bypass it closes.""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -977,7 +988,7 @@ def _rule_command_substitution_content( # check would only change how often an empty-content recursive # `classify()` call is skipped, never a real verdict. if inner_text.strip(): - inner_verdict = classify(inner_text, name_to_value, name_to_raw_value) + inner_verdict = classify(inner_text, name_to_value, name_to_raw_value, name_to_raw_value_git_biased) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) if inner_verdict.deny: @@ -991,7 +1002,9 @@ def _rule_command_substitution_content( if span_end is not None: inner_tokens = tokens[i + 2 : span_end - 1] if inner_tokens: - inner_verdict = _classify_tokens(inner_tokens, name_to_value, name_to_raw_value) + inner_verdict = _classify_tokens( + inner_tokens, name_to_value, name_to_raw_value, name_to_raw_value_git_biased + ) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) if inner_verdict.deny: @@ -1589,6 +1602,78 @@ def _assigned_raw_values(tokens: list[str]) -> dict[str, str]: return values +def _assigned_raw_values_biased_toward(tokens: list[str], literal: str) -> dict[str, str]: + """Like `_assigned_raw_values`, but once a name is assigned LITERAL + (case-insensitively) at ANY point among TOKENS, that name STAYS + LITERAL here regardless of any later, different reassignment -- + unlike `_assigned_raw_values`'s own plain last-occurrence-in-token- + order-wins collapse, which has no concept of which assignment is + actually in effect at bash's own real, sequential runtime relative to + a specific point of use. A name never assigned LITERAL anywhere + resolves exactly as `_assigned_raw_values` itself would. + + CRITICAL bug found by independent adversarial review (round 19, issue + #1375) and independently reproduced live: `_find_git_checkout_ + restore`'s own outer git-token-recognition (`_dynamic_token_resolves_ + only_to_literal`) is fed the ordinary, order-blind `_assigned_raw_ + values` dict, so an entirely ordinary shell idiom -- reusing a + variable name for a later, unrelated purpose after it was already + used as `git` -- silently defeats recognition entirely: `TOOL=git; + $TOOL checkout -- dirty.py; TOOL=npm` resolves `$TOOL`'s own dict + entry to `"npm"` (the LAST assignment in token order), even though + `$TOOL` genuinely was `git` at the actual point of use one statement + earlier. Confirmed live end-to-end through the real wrapper against a + scratch repo with a genuinely dirty, tracked `dirty.py`: the control + command (no trailing reassignment) correctly denies with exit 2; the + same command with a trailing `TOOL=npm` wrongly allows with exit 0, + and actually running it afterward silently discards the uncommitted + edit. Identically reproducible for the command-substitution path + (`G=git; x=$($G checkout -- dirty.py); G=notgit`) and for `restore`. + This is not itself a new gap -- `_assigned_raw_values`'s own + order-blind collapse is a pre-existing, whole-module primitive used + since this module's own earliest rounds, and the module already + discloses the identical class of gap for `$IFS` reassignment + elsewhere as an accepted limitation -- but the checkout/restore + surface is the one consumer where a silent miss means real, + irreversible data loss rather than a missed advisory warning, so a + narrow, targeted mitigation is applied here rather than left + disclosed-only, matching the same "when a token-level ambiguity + cannot be soundly resolved, prefer extra scrutiny over a silent miss" + posture this module already applies pervasively elsewhere (e.g. + `_dynamic_word_may_resolve_to_a_cwd_relocator`'s own OR-based fail- + closed choice for the analogous cd/pushd/popd question). + + Deliberately NOT full execution-order tracking (a far larger, + Stage-2-class change -- this module's own header docstring already + scopes real bash execution semantics out of Stage 1's static token + analysis): this is a bounded, one-directional bias toward the single + safe-to-over-recognize reading (an unrelated tool's own dynamic + `checkout`/`restore`-shaped subcommand at worst triggers a spurious, + reversible live `git diff` check and possible false deny -- the same + safe direction every other ambiguity in this module resolves toward + -- never toward silently missing a real git invocation, which is the + unsafe direction here). Used ONLY to feed the outer git-token- + recognition fallback in `_find_git_checkout_restore` -- every other + consumer of `name_to_raw_value` in this module keeps using the + ordinary, order-blind `_assigned_raw_values` unchanged, since a + reassignment-ambiguity miss elsewhere in this module risks a missed + advisory warning or an unrecognized non-destructive write, not + irreversible data loss.""" + values: dict[str, str] = {} + literal_lower = literal.lower() + for token in tokens: + if _is_dynamic(token): + continue + match = _ASSIGN_RE.match(token) + if not match: + continue + name = match.group(1) + if values.get(name, "").lower() == literal_lower: + continue + values[name] = match.group(2) + return values + + def _strip_leading_assignments(seg: list[str]) -> list[str]: """Bash's own simple-command grammar lets zero or more `NAME=value` environment-assignment tokens precede the actual command word (`X=foo @@ -2263,7 +2348,10 @@ def _strip_array_literal_newlines(tokens: list[str]) -> list[str]: def _rule_array_literal_content( - tokens: list[str], name_to_value: dict[str, str], name_to_raw_value: dict[str, str] + tokens: list[str], + name_to_value: dict[str, str], + name_to_raw_value: dict[str, str], + name_to_raw_value_git_biased: dict[str, str], ) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `NAME=(...)` array-literal span's OWN inner content through this module's full rule set -- bash genuinely @@ -2366,7 +2454,14 @@ def _rule_array_literal_content( the same way `_array_literal_token_span` itself does), so a newline genuinely nested inside a `$(...)`/`(...)` construct WITHIN the array's own inner content (still a real command list there) is left - untouched for the recursive call to classify correctly.""" + untouched for the recursive call to classify correctly. + + NAME_TO_RAW_VALUE_GIT_BIASED (round 19, issue #1375) is threaded + through the recursive `_classify_tokens` call below alongside NAME_TO_ + VALUE/NAME_TO_RAW_VALUE, mirroring `_rule_command_substitution_ + content`'s own identical parameter exactly -- see `_find_git_ + checkout_restore`'s own docstring for what it means and the live + bypass it closes.""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -2383,7 +2478,9 @@ def _rule_array_literal_content( if collapsed and collapsed != inner: readings.append((collapsed, " once its own leading unassigned reference(s) word-split away")) for reading, suffix in readings: - reading_verdict = _classify_tokens(reading, name_to_value, name_to_raw_value) + reading_verdict = _classify_tokens( + reading, name_to_value, name_to_raw_value, name_to_raw_value_git_biased + ) is_git_push = is_git_push or reading_verdict.is_git_push checkout_restore_paths.extend(reading_verdict.checkout_restore_paths) if reading_verdict.deny: @@ -3815,7 +3912,9 @@ def _dynamic_token_resolves_only_to_literal(token: str, name_to_raw_value: dict[ return all(candidate.lower() == literal for candidate in candidates) -def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str]) -> tuple[str | None, list[str], bool]: +def _find_git_checkout_restore( + seg: list[str], name_to_raw_value: dict[str, str], name_to_raw_value_git_biased: dict[str, str] +) -> tuple[str | None, list[str], bool]: """Scan SEG (already assignment-stripped, see `_strip_leading_ assignments`) for a `git checkout`/`git restore` invocation, skipping past git's own global value-taking options the same way @@ -3887,11 +3986,34 @@ def _find_git_checkout_restore(seg: list[str], name_to_raw_value: dict[str, str] `git > /dev/null checkout -- dirty.py` (fully literal, no dynamic content at all) resolved to an empty, wrong `checkout_restore_paths` before this fix, since the redirect operator token itself was - mistaken for the subcommand position and the scan gave up there.""" + mistaken for the subcommand position and the scan gave up there. + + NAME_TO_RAW_VALUE_GIT_BIASED is a second reading, alongside NAME_TO_ + RAW_VALUE, tried for a DYNAMIC `tok` that the ordinary reading + declines -- CRITICAL bug found by independent adversarial review + (round 19, issue #1375) and independently reproduced live: + NAME_TO_RAW_VALUE alone is `_assigned_raw_values`'s own order-blind, + last-occurrence-wins collapse, so `TOOL=git; $TOOL checkout -- + dirty.py; TOOL=npm` -- reusing a variable name for a later, unrelated + purpose, an entirely ordinary idiom, not an exotic construction -- + resolved `$TOOL` to `"npm"` even though it genuinely was `git` at the + actual point of use, silently defeating recognition entirely. + NAME_TO_RAW_VALUE_GIT_BIASED (`_assigned_raw_values_biased_toward`, + see its own docstring) instead reads as `git` whenever `git` was + EVER assigned to that name anywhere in the command, so this second + attempt still recognizes the `git` occurrence. Only ever WIDENS + recognition (an unrelated tool's own dynamic `checkout`/`restore`- + shaped subcommand at worst triggers a spurious, reversible live + `git diff` check), never narrows it -- trying NAME_TO_RAW_VALUE + first, unchanged, preserves this function's own existing "ambiguous + declines" posture for every case that was already resolvable.""" n = len(seg) for i, tok in enumerate(seg): if _is_dynamic(tok): - if not _dynamic_token_resolves_only_to_literal(tok, name_to_raw_value, "git"): + if not ( + _dynamic_token_resolves_only_to_literal(tok, name_to_raw_value, "git") + or _dynamic_token_resolves_only_to_literal(tok, name_to_raw_value_git_biased, "git") + ): continue elif tok.lower() != "git": continue @@ -4063,7 +4185,7 @@ def _first_surviving_segment_word(seg: list[str], name_to_raw_value: dict[str, s def _rule_git_checkout_restore( - segments: list[list[str]], raw_assigned: dict[str, str] + segments: list[list[str]], raw_assigned: dict[str, str], raw_assigned_git_biased: dict[str, str] ) -> tuple[str | None, tuple[str, ...]]: """Extract every `checkout_restore_paths` candidate across every segment of one command, denying outright on any segment where this @@ -4145,11 +4267,18 @@ def _rule_git_checkout_restore( reproduction. The literal scan above is unaffected: it already checks every token in the segment regardless of position, so a literal `cd`/`pushd`/`popd` sitting after a vanishing decoy was already - covered.""" + covered. + + RAW_ASSIGNED_GIT_BIASED is passed straight through as `_find_git_ + checkout_restore`'s own third argument -- see that function's own + docstring for what it means and the live bypass it closes (round 19, + issue #1375).""" saw_cd = False all_paths: list[str] = [] for seg in segments: - subcommand, tokens_after, saw_tree_relocation = _find_git_checkout_restore(seg, raw_assigned) + subcommand, tokens_after, saw_tree_relocation = _find_git_checkout_restore( + seg, raw_assigned, raw_assigned_git_biased + ) if subcommand is None: first = _first_surviving_segment_word(seg, raw_assigned) if any(not _is_dynamic(t) and t in _CWD_RELOCATING_COMMANDS for t in seg) or ( @@ -4376,6 +4505,7 @@ def classify( command: str, outer_name_to_value: dict[str, str] | None = None, outer_name_to_raw_value: dict[str, str] | None = None, + outer_name_to_raw_value_git_biased: dict[str, str] | None = None, ) -> Verdict: """Classify one Bash tool_input.command string. Fails closed (deny) on anything shlex cannot tokenize -- an unparseable command is exactly the @@ -4390,18 +4520,24 @@ def classify( substitution_content`'s own recursive call (issue #1375, eighteenth round) is the one caller that supplies them, so a quoted/fused `$(...)` span's own inner content can resolve a variable assigned - OUTSIDE the span against the same shell scope real bash would use.""" + OUTSIDE the span against the same shell scope real bash would use. + + OUTER_NAME_TO_RAW_VALUE_GIT_BIASED (round 19, issue #1375) is the + same recursive call's own analogous third argument -- see `_classify_ + tokens`'s own docstring and `_find_git_checkout_restore`'s own + docstring for what it means and the live bypass it closes.""" try: tokens = tokenize(command) except TokenizeError as error: return Verdict(True, f"the command could not be parsed as shell syntax ({error}). Failing closed", False) - return _classify_tokens(tokens, outer_name_to_value, outer_name_to_raw_value) + return _classify_tokens(tokens, outer_name_to_value, outer_name_to_raw_value, outer_name_to_raw_value_git_biased) def _classify_tokens( tokens: list[str], outer_name_to_value: dict[str, str] | None = None, outer_name_to_raw_value: dict[str, str] | None = None, + outer_name_to_raw_value_git_biased: dict[str, str] | None = None, ) -> Verdict: """The token-level core of `classify` -- split out so `_rule_command_ substitution_content` can recurse into a `$(...)` span's own inner @@ -4430,20 +4566,37 @@ def _classify_tokens( substitution fix -- for the live bypass each closes) -- `None` (the top-level `classify` entry point, when its own caller has no outer scope of its own to supply) preserves this function's behavior for a - genuinely top-level command exactly.""" + genuinely top-level command exactly. + + OUTER_NAME_TO_RAW_VALUE_GIT_BIASED (round 19, issue #1375) is a third, + parallel outer-scope argument, merged with TOKENS's own `_assigned_ + raw_values_biased_toward(tokens, "git")` (see that function's own + docstring) the same way OUTER_NAME_TO_RAW_VALUE is merged with the + plain `_assigned_raw_values(tokens)` above -- threaded into the same + two recursive calls, AND into `_rule_git_checkout_restore`'s own + third argument below, so a `git` token that is reassigned to + something else LATER in the same command (in an outer scope, inside + a `$(...)`/array-literal span, or both) still resolves to `git` for + checkout/restore recognition specifically. See `_find_git_checkout_ + restore`'s own docstring for the live bypass this closes.""" outer_literals = outer_name_to_value or {} outer_raw = outer_name_to_raw_value or {} + outer_raw_git_biased = outer_name_to_raw_value_git_biased or {} merged_name_to_value = {**outer_literals, **_assigned_literals(tokens)} merged_name_to_raw_value = {**outer_raw, **_assigned_raw_values(tokens)} + merged_name_to_raw_value_git_biased = { + **outer_raw_git_biased, + **_assigned_raw_values_biased_toward(tokens, "git"), + } content_reason, content_is_git_push, content_checkout_restore_paths = _rule_command_substitution_content( - tokens, merged_name_to_value, merged_name_to_raw_value + tokens, merged_name_to_value, merged_name_to_raw_value, merged_name_to_raw_value_git_biased ) if content_reason: return Verdict(True, content_reason, content_is_git_push, content_checkout_restore_paths) array_content_reason, array_content_is_git_push, array_content_checkout_restore_paths = _rule_array_literal_content( - tokens, merged_name_to_value, merged_name_to_raw_value + tokens, merged_name_to_value, merged_name_to_raw_value, merged_name_to_raw_value_git_biased ) is_git_push = content_is_git_push or array_content_is_git_push checkout_restore_paths = content_checkout_restore_paths + array_content_checkout_restore_paths @@ -4454,6 +4607,7 @@ def _classify_tokens( segments = [s for s in (_strip_leading_assignments(seg) for seg in segment_tokens(tokens)) if s] assigned = {**outer_literals, **_assigned_literals(tokens)} raw_assigned = {**outer_raw, **_assigned_raw_values(tokens)} + raw_assigned_git_biased = {**outer_raw_git_biased, **_assigned_raw_values_biased_toward(tokens, "git")} lowered_command = " ".join(tokens).lower() is_git_push = is_git_push or any(_is_git_push_segment(seg, raw_assigned) for seg in segments) @@ -4485,7 +4639,9 @@ def _classify_tokens( checkout_restore_paths, ) - own_checkout_restore_hit, own_checkout_restore_paths = _rule_git_checkout_restore(segments, raw_assigned) + own_checkout_restore_hit, own_checkout_restore_paths = _rule_git_checkout_restore( + segments, raw_assigned, raw_assigned_git_biased + ) checkout_restore_paths = checkout_restore_paths + own_checkout_restore_paths if own_checkout_restore_hit: return Verdict(True, own_checkout_restore_hit, is_git_push, checkout_restore_paths) diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index a2612be9..db575ebe 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1912,6 +1912,50 @@ def test_checkout_denied_when_a_dynamic_git_token_sits_inside_a_quoted_command_s assert result.returncode == 2, f"stderr={result.stderr!r}" +def test_checkout_denied_when_a_dynamic_git_token_is_reassigned_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-19 independent review, issue + #1375). `_assigned_raw_values`'s own order-blind, last-occurrence- + wins collapse meant an entirely ordinary shell idiom -- reusing a + variable name for a later, unrelated purpose after it was already + used as `git` -- silently defeated recognition entirely, since the + variable's own dict entry resolved to the LATER value, not the one + genuinely in effect at the actual point of use. Live-verified before + this fix: `TOOL=git; $TOOL checkout -- dirty.py; TOOL=npm` was + wrongly allowed outright through the real wrapper, and the real + command afterward silently discarded a genuinely dirty `dirty.py`.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("TOOL=git; $TOOL checkout -- dirty.py; TOOL=npm", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_restore_denied_when_a_dynamic_git_token_is_reassigned_after_use(tmp_path: Path) -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-19 finding was confirmed live for both subcommands.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("TOOL=git; $TOOL restore dirty.py; TOOL=npm", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_checkout_denied_when_a_dynamic_git_token_inside_a_command_substitution_is_reassigned_after_use( + tmp_path: Path, +) -> None: + """Companion to the two pins above, for the command-substitution + shape: the SAME reassignment-after-use gap, reached through + `_rule_command_substitution_content`'s own outer-scope threading + (round 18). Live-verified before this fix: `G=git; x=$($G checkout + -- dirty.py); G=notgit` was wrongly allowed outright through the real + wrapper.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("G=git; x=$($G checkout -- dirty.py); G=notgit", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 0467ac9f..d3e62a23 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -1358,7 +1358,7 @@ def test_rule_command_substitution_content_detects_an_embedded_install(tool: str a punctuation character shlex breaks a word at, so an assignment's `NAME=` prefix stays fused onto the leading `$` in the same token.""" tokens = ["x=$", "(", tool, "install", "evil-pkg", ")"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}) assert reason is not None @@ -1374,7 +1374,7 @@ def test_rule_command_substitution_content_allows_harmless_inner_content(value: silently dropping a non-denying inner `is_git_push=True` signal (see the function's own docstring).""" tokens = ["echo", "$", "(", "date", value, ")"] - assert checker._rule_command_substitution_content(tokens, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}) == (None, False, ()) # --- Issue #1326 Stage 1, fifteenth round: bash's own leading-assignment ---- @@ -1493,7 +1493,7 @@ def test_rule_array_literal_content_detects_a_denied_pair_regardless_of_a_leadin `Y=1; A=(uv install $Y); "${A[@]}"` was wrongly ALLOWED before this function existed.""" tokens = ["dummy=", "(", f"${first}", "uv", "install", f"${second}", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) assert reason is not None @@ -1512,7 +1512,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_bare_ref(unse fused with other text (not a bare whole-token reference), must NOT be collapsed -- that shape does not word-split away to nothing.""" tokens = ["dummy=", "(", f"${unset_name}", verb_a, "install", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) assert reason is not None @@ -1532,13 +1532,13 @@ def test_rule_array_literal_content_allows_harmless_content() -> None: denied pattern, with or without a leading unassigned reference, stays allowed.""" tokens = ["dummy=", "(", "$NEVERSET", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_no_span_present() -> None: """Robustness: a token stream with no array-literal span at all (e.g. an ordinary command) returns cleanly, never a crash.""" - assert checker._rule_array_literal_content(["echo", "hi"], {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(["echo", "hi"], {}, {}, {}) == (None, False, ()) def test_strip_leading_unassigned_bare_refs_stops_at_a_fused_token() -> None: @@ -1564,7 +1564,7 @@ def test_rule_array_literal_content_empty_array_is_harmless() -> None: """No false positive / no crash: an empty array literal `NAME=()` has no inner content to recursively classify at all.""" tokens = ["dummy=", "(", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leading_unassigned_ref() -> None: @@ -1573,7 +1573,7 @@ def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leadin `_strip_leading_unassigned_bare_refs` to strip -- the collapsed reading equals the as-is one, so only one classification is needed.""" tokens = ["dummy=", "(", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> None: @@ -1590,7 +1590,7 @@ def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> No real, with a dynamic verb argument right after it -- exactly B2's own watched shape.""" tokens = ["dummy=", "(", "$NEVERSET", "uv", "$VERB", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) assert reason is not None assert "unassigned reference" in reason @@ -1632,7 +1632,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_braced_bare_r this round, silently degrading the collapsed reading to a no-op for this shape.""" tokens = ["dummy=", "(", f"${{{unset_name}}}", verb_a, "install", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) assert reason is not None @@ -1647,7 +1647,7 @@ def test_rule_array_literal_content_detects_an_outer_scope_resolved_pair() -> No recursive `_classify_tokens` call.""" tokens = ["dummy=", "(", "$G", "$P", "$M", ")"] outer = {"G": "gh", "P": "pr", "M": "merge"} - reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer) + reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer, outer) assert reason is not None @@ -1683,7 +1683,7 @@ def test_rule_command_substitution_content_detects_an_outer_scope_resolved_check for the array-literal span.""" tokens = ["x=$", "(", "$G", "checkout", "--", "dirty.py", ")"] outer = {"G": "git"} - reason, _, checkout_restore_paths = checker._rule_command_substitution_content(tokens, outer, outer) + reason, _, checkout_restore_paths = checker._rule_command_substitution_content(tokens, outer, outer, outer) assert reason is None assert checkout_restore_paths == ("dirty.py",) @@ -1719,6 +1719,104 @@ def test_classify_extracts_quoted_command_substitution_checkout_paths_with_outer assert verdict.checkout_restore_paths == ("dirty.py",) +def test_assigned_raw_values_biased_toward_stays_on_literal_once_assigned() -> None: + """Model-based, regression pin for the real bypass found live by Step + 8 independent review, nineteenth round (issue #1375): once a name is + assigned the biased-toward literal at any point, a LATER, different + reassignment of the SAME name must not overwrite it back out -- + `TOOL=git; ...; TOOL=npm` must still resolve `TOOL` to `git` here, + unlike `_assigned_raw_values`'s own plain last-occurrence-wins + collapse.""" + assert checker._assigned_raw_values_biased_toward(["TOOL=git", "TOOL=npm"], "git") == {"TOOL": "git"} + + +def test_assigned_raw_values_biased_toward_locks_on_regardless_of_order() -> None: + """The literal-assignment can arrive BEFORE or AFTER a different + reassignment of the same name and the end result is the same -- this + function does not attempt real execution-order tracking, only a + bounded "was LITERAL ever assigned to this name" bias.""" + assert checker._assigned_raw_values_biased_toward(["TOOL=npm", "TOOL=git"], "git") == {"TOOL": "git"} + + +def test_assigned_raw_values_biased_toward_falls_back_to_last_assignment_when_literal_never_seen() -> None: + """A name never assigned the biased-toward literal anywhere resolves + exactly as `_assigned_raw_values`'s own plain last-occurrence-wins + collapse would -- this function only ever WIDENS toward the literal, + never changes behavior for a name that was never a candidate.""" + assert checker._assigned_raw_values_biased_toward(["TOOL=npm", "TOOL=yarn"], "git") == {"TOOL": "yarn"} + + +@_PROPERTIES +@given(name=_IDENTIFIERS, decoy_value=_VALUES, tail=st.lists(_IDENTIFIERS, max_size=2)) +def test_assigned_raw_values_biased_toward_matches_plain_collapse_when_never_reassigned_to_literal( + name: str, decoy_value: str, tail: list[str] +) -> None: + """Model-based: for a single assignment never matching the biased- + toward literal, `_assigned_raw_values_biased_toward` agrees exactly + with the plain, order-blind `_assigned_raw_values`.""" + assume(decoy_value.lower() != "git") + tokens = [f"{name}={decoy_value}", *tail] + assert checker._assigned_raw_values_biased_toward(tokens, "git") == checker._assigned_raw_values(tokens) + + +def test_find_git_checkout_restore_recognizes_a_git_biased_reassigned_token() -> None: + """Model-based, regression pin for the real bypass found live by Step + 8 independent review, nineteenth round (issue #1375): the ordinary + NAME_TO_RAW_VALUE reading declines (TOOL's own collapsed value is + "npm", not "git"), but the GIT_BIASED reading recognizes `git` was + assigned to TOOL at some point, so the occurrence is still found.""" + seg = ["$TOOL", "checkout", "--", "f.py"] + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore( + seg, {"TOOL": "npm"}, {"TOOL": "git"} + ) + assert subcommand == "checkout" + assert tokens_after == ["--", "f.py"] + assert saw_tree_relocation is False + + +def test_find_git_checkout_restore_still_declines_when_neither_reading_resolves_to_git() -> None: + """No false positive: when NEITHER the ordinary nor the git-biased + reading resolves `tok` to `git`, the occurrence is still declined -- + the git-biased reading only ever widens recognition, never invents a + match out of nothing.""" + seg = ["$TOOL", "checkout", "--", "f.py"] + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(seg, {"TOOL": "svn"}, {"TOOL": "svn"}) + assert subcommand is None + + +def test_classify_extracts_checkout_paths_when_git_token_is_reassigned_after_use() -> None: + """End-to-end regression pin for the round-19 finding at the + `classify()` level, top-level shape (no command substitution needed + at all) -- an entirely ordinary "reuse a variable name for a later, + unrelated purpose" idiom. Confirmed live before this fix: + `TOOL=git; $TOOL checkout -- dirty.py; TOOL=npm` resolved to an EMPTY + `checkout_restore_paths`, even though `$TOOL` genuinely was `git` at + its actual point of use.""" + verdict = checker.classify("TOOL=git; $TOOL checkout -- dirty.py; TOOL=npm") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + +def test_classify_extracts_restore_paths_when_git_token_is_reassigned_after_use() -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-19 finding was confirmed live for both subcommands.""" + verdict = checker.classify("TOOL=git; $TOOL restore dirty.py; TOOL=npm") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + +def test_classify_extracts_checkout_paths_when_git_token_is_reassigned_after_a_command_substitution() -> None: + """End-to-end regression pin for the round-19 finding's command- + substitution shape: the SAME reassignment-after-use gap, reached + through `_rule_command_substitution_content`'s own outer-scope + threading (round 18). Confirmed live before this fix: `G=git; + x=$($G checkout -- dirty.py); G=notgit` resolved to an EMPTY + `checkout_restore_paths`.""" + verdict = checker.classify("G=git; x=$($G checkout -- dirty.py); G=notgit") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + @_PROPERTIES @given(name=_IDENTIFIERS, subscript=st.sampled_from(["0", "1", "@", "*", "$i"])) def test_token_is_all_unassigned_refs_recognizes_a_braced_subscript(name: str, subscript: str) -> None: @@ -1793,7 +1891,7 @@ def test_rule_array_literal_content_detects_a_braced_subscript_decoy() -> None: the subscript decoy blocked it from ever firing until it collapsed away.""" tokens = ["dummy=", "(", "${NEVERSET[0]}", "uv", "$VERB", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) assert reason is not None @@ -1804,7 +1902,7 @@ def test_rule_array_literal_content_detects_a_fused_reference_chain_decoy() -> N before a fused chain of two bare references was recognized as vanishing as a unit.""" tokens = ["dummy=", "(", "$A_UNSET$B_UNSET", "gh", "pr", "merge", "1", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) assert reason is not None @@ -1937,7 +2035,7 @@ def test_rule_command_substitution_content_scans_second_fused_span_in_same_token this test only proves that fix reached end-to-end through `_rule_command_substitution_content`'s own scan loop.""" tokens = ["echo", "$(echo ok)$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}) assert reason is not None @@ -1946,20 +2044,20 @@ def test_rule_command_substitution_content_skips_blank_fused_span_then_finds_den skipped without denying by itself, but scanning continues to the next fused span in the same token.""" tokens = ["echo", "$( )$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}) assert reason is not None def test_rule_command_substitution_content_both_fused_spans_harmless() -> None: tokens = ["echo", "$(echo ok)$(echo also-ok)"] - assert checker._rule_command_substitution_content(tokens, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}) == (None, False, ()) def test_rule_command_substitution_content_empty_unquoted_span_skipped() -> None: """An empty, unquoted `$()` substitution has no inner tokens to recurse into -- distinct from the fused/quoted empty-span case above.""" tokens = ["$", "(", ")"] - assert checker._rule_command_substitution_content(tokens, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}) == (None, False, ()) def test_tokenize_raises_on_unbalanced_quote() -> None: @@ -2913,7 +3011,7 @@ def test_find_git_checkout_restore_none_when_only_global_flags_and_no_subcommand checkout/restore invocation -- the while loop runs off the end of the segment (`j == n`) rather than finding a literal `checkout`/`restore` token.""" - subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(["git", "-C", "/tmp/x"], {}) + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(["git", "-C", "/tmp/x"], {}, {}) assert subcommand is None assert tokens_after == [] assert saw_tree_relocation is False @@ -2998,7 +3096,7 @@ def test_find_git_checkout_restore_finds_git_at_any_segment_position(prefix: lis found at `seg[0]`.""" assume(all(p != "git" for p in prefix)) seg = [*prefix, "git", "checkout", "--", *paths] - subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}) + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}, {}) assert subcommand == "checkout" assert tokens_after == ["--", *paths] assert saw_tree_relocation is False @@ -3007,7 +3105,7 @@ def test_find_git_checkout_restore_finds_git_at_any_segment_position(prefix: lis def test_find_git_checkout_restore_none_for_a_segment_with_no_git() -> None: """No false positive: a segment with no literal `git` token at all is never treated as a checkout/restore invocation.""" - subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["echo", "checkout", "restore"], {}) + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["echo", "checkout", "restore"], {}, {}) assert subcommand is None @@ -3020,7 +3118,7 @@ def test_find_git_checkout_restore_flags_tree_relocation(flag: str) -> None: rather than let the live wrapper check the wrong tree (issue #1375's own Fact 5 cwd finding).""" seg = ["git", flag, "/some/path", "checkout", "--", "f.py"] - subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}) + subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}, {}) assert subcommand == "checkout" assert saw_tree_relocation is True @@ -3030,7 +3128,7 @@ def test_find_git_checkout_restore_does_not_flag_lowercase_c_config_flag() -> No case-sensitively distinct from `-C` (uppercase, relocates the working tree) and must never be conflated with it (issue #1375's own Fact 5).""" seg = ["git", "-c", "user.name=x", "checkout", "--", "f.py"] - subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}) + subcommand, _tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}, {}) assert subcommand == "checkout" assert saw_tree_relocation is False @@ -3041,7 +3139,7 @@ def test_find_git_checkout_restore_is_a_non_goal_for_a_dynamic_subcommand() -> N shaped and is not detected -- the same disclosed-residual convention this module's own `KNOWN_BYPASS_COMMANDS` test list already uses for the analogous dynamic-tool/dynamic-verb case.""" - subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["git", "$V", "--", "f.py"], {}) + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(["git", "$V", "--", "f.py"], {}, {}) assert subcommand is None @@ -3062,7 +3160,7 @@ def test_find_git_checkout_restore_skips_a_vanishing_decoy_between_git_and_subco closed for `git push` over rounds 20-24 of issue #1326, using the same `_token_is_all_unassigned_refs` primitive this fix now reuses here.""" seg = ["git", "$NEVERSET", "checkout", "--", "file.py"] - subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}) + subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore(seg, {}, {}) assert subcommand == "checkout" assert tokens_after == ["--", "file.py"] assert saw_tree_relocation is False @@ -3192,7 +3290,7 @@ def test_find_git_checkout_restore_does_not_skip_an_assigned_dynamic_token() -> unambiguously vanish, so it still makes this `git` occurrence ambiguous -- unchanged from the pre-fix behavior for this case.""" seg = ["git", "$SET", "checkout", "--", "file.py"] - subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(seg, {"SET": "-C"}) + subcommand, _tokens_after, _saw = checker._find_git_checkout_restore(seg, {"SET": "-C"}, {}) assert subcommand is None @@ -3520,7 +3618,7 @@ def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_pat command (`git checkout -- a.py; git restore b.py`) accumulate paths from every segment, not just the first.""" segments = [["git", "checkout", "--", *command_paths], ["git", "restore", *command_paths]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) assert reason is None assert resolved == (*command_paths, *command_paths) @@ -3533,7 +3631,7 @@ def test_rule_git_checkout_restore_denies_when_git_dir_env_var_assigned() -> Non token-shape fact) rather than letting the live wrapper check the wrong tree.""" segments = [["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}, {}) assert reason is not None assert resolved == () @@ -3544,7 +3642,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_cd() -> Non the wrapper's own fixed `.cwd` unsound for a LATER checkout/restore segment -- denied outright.""" segments = [["cd", "/tmp"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) assert reason is not None assert resolved == () @@ -3554,7 +3652,7 @@ def test_rule_git_checkout_restore_allows_cd_after_the_checkout_segment() -> Non `cd` in an EARLIER segment -- a `cd` AFTER the checkout/restore segment does not retroactively make the already-scanned segment unsound.""" segments = [["git", "checkout", "--", "f.py"], ["cd", "/tmp"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) assert reason is None assert resolved == ("f.py",) @@ -3570,7 +3668,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_pushd_or_po claim that the wrapper's live check then found clean at the wrong `.cwd`, silently allowing a real, uncommitted-change discard.""" segments = [[relocator, "/tmp"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) assert reason is not None assert resolved == () @@ -3595,7 +3693,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_starts_with_a_ `checkout_restore_paths` claim the same way round 9's own fix closed for the literal case.""" segments = [["$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}) assert reason is not None assert resolved == () @@ -3607,7 +3705,7 @@ def test_rule_git_checkout_restore_allows_a_genuinely_vanishing_dynamic_word() - real bash would run whatever token follows as the actual command word instead, and that token is scanned on its own merits.""" segments = [["${NEVERSET}", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) assert reason is None assert resolved == ("f.py",) @@ -3637,7 +3735,7 @@ def test_rule_git_checkout_restore_allows_a_dynamic_word_resolving_to_something_ word's actual candidate value and only flag when it could genuinely be `cd`/`pushd`/`popd`.""" segments = [["$EDITOR", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}, {}) assert reason is None assert resolved == ("f.py",) @@ -3716,7 +3814,7 @@ def test_dynamic_word_may_resolve_to_a_cwd_relocator_true_for_a_still_dynamic_ca def test_rule_git_checkout_restore_denies_a_still_dynamic_candidate() -> None: segments = [["${UNSET:-$OTHER}", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}, {}) assert reason is not None assert resolved == () @@ -3751,7 +3849,7 @@ def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_v assigned) resolved to a CONFIDENT, WRONG `checkout_restore_paths` claim -- real bash genuinely runs `cd sub` there.""" segments = [["$NEVERSET", "$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}) assert reason is not None assert resolved == () @@ -3794,7 +3892,7 @@ def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_r Live-verified before this fix: `X=cd; > /dev/null $X sub; git checkout -- dirty.py` resolved to a confident, wrong ALLOW.""" segments = [[">", "/dev/null", "$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}) assert reason is not None assert resolved == () @@ -3815,7 +3913,7 @@ def test_find_git_checkout_restore_skips_a_redirect_between_git_and_subcommand() Live-verified before this fix: `git > /dev/null checkout -- dirty.py` resolved to an empty, wrong `checkout_restore_paths`.""" subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore( - ["git", ">", "/dev/null", "checkout", "--", "f.py"], {} + ["git", ">", "/dev/null", "checkout", "--", "f.py"], {}, {} ) assert subcommand == "checkout" assert tokens_after == ["--", "f.py"] @@ -3867,7 +3965,7 @@ def test_find_git_checkout_restore_recognizes_a_dynamic_git_token() -> None: `checkout_restore_paths` even though `$G` unambiguously resolves to `git`.""" subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore( - ["$G", "checkout", "--", "f.py"], {"G": "git"} + ["$G", "checkout", "--", "f.py"], {"G": "git"}, {} ) assert subcommand == "checkout" assert tokens_after == ["--", "f.py"] @@ -3879,7 +3977,7 @@ def test_find_git_checkout_restore_declines_an_unresolvable_dynamic_first_word() resolve to `git` (unrelated tool, or unresolvable) is not mistaken for a git invocation.""" subcommand, _tokens_after, _saw_tree_relocation = checker._find_git_checkout_restore( - ["$TOOL", "checkout", "--", "f.py"], {"TOOL": "svn"} + ["$TOOL", "checkout", "--", "f.py"], {"TOOL": "svn"}, {} ) assert subcommand is None @@ -4016,7 +4114,7 @@ def test_redirect_span_length_with_optional_fd_recognizes_a_fused_fd_redirect() def test_find_git_checkout_restore_skips_a_digit_prefixed_redirect_between_git_and_subcommand() -> None: subcommand, tokens_after, saw_tree_relocation = checker._find_git_checkout_restore( - ["git", ">", "out.log", "2", ">&", "1", "checkout", "--", "f.py"], {} + ["git", ">", "out.log", "2", ">&", "1", "checkout", "--", "f.py"], {}, {} ) assert subcommand == "checkout" assert tokens_after == ["--", "f.py"] From a01703465e71fddff63766b0c0c0a5f3912b7359 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 18:05:58 +0000 Subject: [PATCH 25/46] fix(hooks): apply the reassignment bias fix to cd/pushd/popd too A fresh, independent adversarial review of this PR's current head (round 20) found that `_rule_git_checkout_restore`'s own dynamic-cd- relocation check (`_dynamic_word_may_resolve_to_a_cwd_relocator`) was fed only the ordinary, order-blind `raw_assigned` dict -- the IDENTICAL gap round 19 closed for the sibling git-token-recognition consumer in the same function, just left open here. Live-verified: `X=cd; $X sub; git checkout -- dirty.py; X=somethingelse` (reusing a variable name for a later, unrelated purpose, the same ordinary idiom round 19's own finding used) resolved to a CONFIDENT, WRONG `checkout_restore_paths` claim -- `$X` genuinely was `cd` at its actual point of use one statement earlier, but the trailing reassignment made the collapsed dict show `"somethingelse"` instead, so the earlier relocation was silently missed entirely. Confirmed live end-to-end through the real wrapper against a scratch repo with `sub/dirty.py` genuinely dirty relative to `sub`: the control command (no trailing reassignment) correctly denies with exit 2; the same command with a trailing `X=somethingelse` wrongly allowed with exit 0, and actually running it afterward silently discarded the uncommitted edit. Identically reproducible for `pushd`. Fixed the same way round 19 closed the git-token case: - Generalized `_assigned_raw_values_biased_toward(tokens, literal)` to `_assigned_raw_values_biased_toward(tokens, literals: frozenset[str])`, so ONE caller can bias toward several interchangeable candidates at once -- `cd`/`pushd`/`popd` are three different literals that all answer the same "was the working tree possibly relocated" question. Round 19's own single-literal `git` call sites now pass a one-element frozenset (`frozenset({"git"})`); `_CWD_RELOCATING_COMMANDS` itself is now a `frozenset` to match. - `_rule_git_checkout_restore` now also computes `raw_assigned_cd_biased` and tries `_dynamic_word_may_resolve_to_a_cwd_relocator` against it as a fallback when the ordinary `raw_assigned` reading declines. Scoped narrower than round 19's own full recursive-chain threading: `raw_assigned_cd_biased` is built once, at the current `_classify_tokens` invocation's own top-level segments (merged with the plain, non-biased OUTER_RAW), not threaded as a fourth parameter through `classify()`/ `_rule_command_substitution_content`/`_rule_array_literal_content` the way round 19's git-biased dict was. A `cd`/`pushd`/`popd` occurring INSIDE a `$(...)`/array-literal span runs in an isolated subshell and never relocates the OUTER shell's own cwd at real bash runtime regardless of any outer reassignment, so there is no equivalent live bypass to close by threading further -- unlike a `git` token, whose own resolved identity is unaffected by which shell evaluates it. Both subcommands and all three `_CWD_RELOCATING_COMMANDS` members are covered end to end (`hooks/test_gitapex_check_bash_safety.py`, through the real wrapper) and at the unit/property level (`tests/test_gitapex_ check_bash_safety_properties.py`). One pre-existing test (`test_rule_ git_checkout_restore_allows_a_dynamic_word_resolving_to_something_ harmless`) needed its own hand-built `raw_assigned_cd_biased` argument corrected to mirror `raw_assigned` -- in real production use the two dicts are always built from the identical token stream, but `_dynamic_ word_may_resolve_to_a_cwd_relocator`'s own fail-toward-"might be a relocator" posture on an unresolvable name meant an artificially sparse stand-in dict (unlike round 19's git-biased fallback, whose own fail-toward-"not git" posture tolerates an empty stand-in safely) would have wrongly flagged the test's own EDITOR=vim dispatch idiom; confirmed via a direct `classify()` call that the real, non-test code path was never actually affected. Full gate suite green: ruff check/ format, mypy, xenon (CI thresholds), the detection-logic property- coverage gate, and 100% line+branch coverage on hooks/gitapex_check_ bash_safety.py. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 105 +++++++++++++---- hooks/test_gitapex_check_bash_safety.py | 35 ++++++ ...st_gitapex_check_bash_safety_properties.py | 111 +++++++++++++++--- 3 files changed, 211 insertions(+), 40 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index fce43ce2..6880a55e 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -1602,15 +1602,22 @@ def _assigned_raw_values(tokens: list[str]) -> dict[str, str]: return values -def _assigned_raw_values_biased_toward(tokens: list[str], literal: str) -> dict[str, str]: - """Like `_assigned_raw_values`, but once a name is assigned LITERAL - (case-insensitively) at ANY point among TOKENS, that name STAYS - LITERAL here regardless of any later, different reassignment -- - unlike `_assigned_raw_values`'s own plain last-occurrence-in-token- - order-wins collapse, which has no concept of which assignment is - actually in effect at bash's own real, sequential runtime relative to - a specific point of use. A name never assigned LITERAL anywhere - resolves exactly as `_assigned_raw_values` itself would. +def _assigned_raw_values_biased_toward(tokens: list[str], literals: frozenset[str]) -> dict[str, str]: + """Like `_assigned_raw_values`, but once a name is assigned a value + that is a member of LITERALS (case-insensitively) at ANY point among + TOKENS, that name STAYS on that member here regardless of any later, + different reassignment -- unlike `_assigned_raw_values`'s own plain + last-occurrence-in-token-order-wins collapse, which has no concept of + which assignment is actually in effect at bash's own real, sequential + runtime relative to a specific point of use. A name never assigned + any member of LITERALS anywhere resolves exactly as `_assigned_raw_ + values` itself would. LITERALS is a set rather than a single string + so one caller can bias toward several interchangeable candidates at + once (round 20, issue #1375: `_CWD_RELOCATING_COMMANDS` -- `cd`, + `pushd`, `popd` are three DIFFERENT literals that all answer the same + "was the working tree possibly relocated" question, see this + function's own second CRITICAL-bug paragraph below); round 19's own + single-literal `git` caller passes a one-element set. CRITICAL bug found by independent adversarial review (round 19, issue #1375) and independently reproduced live: `_find_git_checkout_ @@ -1652,15 +1659,52 @@ def _assigned_raw_values_biased_toward(tokens: list[str], literal: str) -> dict[ reversible live `git diff` check and possible false deny -- the same safe direction every other ambiguity in this module resolves toward -- never toward silently missing a real git invocation, which is the - unsafe direction here). Used ONLY to feed the outer git-token- - recognition fallback in `_find_git_checkout_restore` -- every other - consumer of `name_to_raw_value` in this module keeps using the - ordinary, order-blind `_assigned_raw_values` unchanged, since a - reassignment-ambiguity miss elsewhere in this module risks a missed + unsafe direction here). Used to feed the outer git-token-recognition + fallback in `_find_git_checkout_restore`, AND (round 20, below) the + cd/pushd/popd-relocation fallback in `_rule_git_checkout_restore` -- + every OTHER consumer of `name_to_raw_value` in this module keeps + using the ordinary, order-blind `_assigned_raw_values` unchanged, + since a reassignment-ambiguity miss for one of those risks a missed advisory warning or an unrecognized non-destructive write, not - irreversible data loss.""" + irreversible data loss -- the same reasoning that scoped round 19's + own original, narrower fix. + + CRITICAL bug found by independent adversarial review (round 20, issue + #1375) and independently reproduced live: `_rule_git_checkout_ + restore`'s own dynamic-cd-relocation check + (`_dynamic_word_may_resolve_to_a_cwd_relocator`) was fed only the + ordinary, order-blind `raw_assigned` -- the IDENTICAL gap round 19 + closed for the sibling git-token-recognition consumer in the same + function, just left open here. `X=cd; $X sub; git checkout -- + dirty.py; X=somethingelse` (reusing a variable name for a later, + unrelated purpose, the same ordinary idiom round 19's own finding + used) resolved to a CONFIDENT, WRONG `checkout_restore_paths` claim + -- `$X` genuinely was `cd` at its actual point of use one statement + earlier, but the trailing reassignment made the collapsed dict show + `"somethingelse"` instead, so the earlier relocation was silently + missed entirely. Confirmed live end-to-end through the real wrapper + against a scratch repo with `sub/dirty.py` genuinely dirty relative + to `sub`: the control command (no trailing reassignment) correctly + denies with exit 2; the same command with a trailing + `X=somethingelse` wrongly allows with exit 0, and actually running it + afterward silently discarded the uncommitted edit. Identically + reproducible for `pushd`. Closed the same way round 19 closed the + git-token case: `_rule_git_checkout_restore`'s own cd-relocation + check now also tries `_dynamic_word_may_resolve_to_a_cwd_relocator` + against a `raw_assigned_cd_biased` reading -- `_assigned_raw_values_ + biased_toward(tokens, _CWD_RELOCATING_COMMANDS)` -- as a fallback + when the ordinary reading declines. Scoped to the current + `_classify_tokens` invocation's own top-level segments only (merged + with plain OUTER_RAW, not a THIRD `..._cd_biased` outer-scope + parameter threaded through the whole recursive chain the way round + 19's git-biased fix was): a `cd`/`pushd`/`popd` occurring INSIDE a + `$(...)`/array-literal span runs in an isolated subshell and never + relocates the OUTER shell's own cwd at real bash runtime regardless + of any outer reassignment, so there is no equivalent live bypass to + close by threading further -- unlike a `git` token, whose own + resolved identity is unaffected by which shell (sub- or otherwise) + evaluates it.""" values: dict[str, str] = {} - literal_lower = literal.lower() for token in tokens: if _is_dynamic(token): continue @@ -1668,7 +1712,7 @@ def _assigned_raw_values_biased_toward(tokens: list[str], literal: str) -> dict[ if not match: continue name = match.group(1) - if values.get(name, "").lower() == literal_lower: + if values.get(name, "").lower() in literals: continue values[name] = match.group(2) return values @@ -4055,7 +4099,7 @@ def _find_git_checkout_restore( return None, [], False -_CWD_RELOCATING_COMMANDS = {"cd", "pushd", "popd"} +_CWD_RELOCATING_COMMANDS = frozenset({"cd", "pushd", "popd"}) def _dynamic_word_may_resolve_to_a_cwd_relocator(token: str, name_to_raw_value: dict[str, str]) -> bool: @@ -4185,7 +4229,10 @@ def _first_surviving_segment_word(seg: list[str], name_to_raw_value: dict[str, s def _rule_git_checkout_restore( - segments: list[list[str]], raw_assigned: dict[str, str], raw_assigned_git_biased: dict[str, str] + segments: list[list[str]], + raw_assigned: dict[str, str], + raw_assigned_git_biased: dict[str, str], + raw_assigned_cd_biased: dict[str, str], ) -> tuple[str | None, tuple[str, ...]]: """Extract every `checkout_restore_paths` candidate across every segment of one command, denying outright on any segment where this @@ -4272,7 +4319,10 @@ def _rule_git_checkout_restore( RAW_ASSIGNED_GIT_BIASED is passed straight through as `_find_git_ checkout_restore`'s own third argument -- see that function's own docstring for what it means and the live bypass it closes (round 19, - issue #1375).""" + issue #1375). RAW_ASSIGNED_CD_BIASED feeds a SECOND fallback, for the + cd/pushd/popd-relocation check just below (round 20, issue #1375) -- + see `_assigned_raw_values_biased_toward`'s own second CRITICAL-bug + paragraph for the live bypass this one closes.""" saw_cd = False all_paths: list[str] = [] for seg in segments: @@ -4284,7 +4334,10 @@ def _rule_git_checkout_restore( if any(not _is_dynamic(t) and t in _CWD_RELOCATING_COMMANDS for t in seg) or ( first is not None and _is_dynamic(first) - and _dynamic_word_may_resolve_to_a_cwd_relocator(first, raw_assigned) + and ( + _dynamic_word_may_resolve_to_a_cwd_relocator(first, raw_assigned) + or _dynamic_word_may_resolve_to_a_cwd_relocator(first, raw_assigned_cd_biased) + ) ): saw_cd = True continue @@ -4586,7 +4639,7 @@ def _classify_tokens( merged_name_to_raw_value = {**outer_raw, **_assigned_raw_values(tokens)} merged_name_to_raw_value_git_biased = { **outer_raw_git_biased, - **_assigned_raw_values_biased_toward(tokens, "git"), + **_assigned_raw_values_biased_toward(tokens, frozenset({"git"})), } content_reason, content_is_git_push, content_checkout_restore_paths = _rule_command_substitution_content( @@ -4607,7 +4660,11 @@ def _classify_tokens( segments = [s for s in (_strip_leading_assignments(seg) for seg in segment_tokens(tokens)) if s] assigned = {**outer_literals, **_assigned_literals(tokens)} raw_assigned = {**outer_raw, **_assigned_raw_values(tokens)} - raw_assigned_git_biased = {**outer_raw_git_biased, **_assigned_raw_values_biased_toward(tokens, "git")} + raw_assigned_git_biased = { + **outer_raw_git_biased, + **_assigned_raw_values_biased_toward(tokens, frozenset({"git"})), + } + raw_assigned_cd_biased = {**outer_raw, **_assigned_raw_values_biased_toward(tokens, _CWD_RELOCATING_COMMANDS)} lowered_command = " ".join(tokens).lower() is_git_push = is_git_push or any(_is_git_push_segment(seg, raw_assigned) for seg in segments) @@ -4640,7 +4697,7 @@ def _classify_tokens( ) own_checkout_restore_hit, own_checkout_restore_paths = _rule_git_checkout_restore( - segments, raw_assigned, raw_assigned_git_biased + segments, raw_assigned, raw_assigned_git_biased, raw_assigned_cd_biased ) checkout_restore_paths = checkout_restore_paths + own_checkout_restore_paths if own_checkout_restore_hit: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index db575ebe..b0aadc98 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1956,6 +1956,41 @@ def test_checkout_denied_when_a_dynamic_git_token_inside_a_command_substitution_ assert result.returncode == 2, f"stderr={result.stderr!r}" +def test_checkout_denied_when_a_dynamic_cd_relocator_is_reassigned_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-20 independent review, issue + #1375). `_rule_git_checkout_restore`'s own dynamic-cd-relocation + check was fed only the ordinary, order-blind `raw_assigned` -- the + IDENTICAL reassignment-after-use gap round 19 closed for the sibling + git-token-recognition consumer in the same function, just left open + here. Live-verified before this fix: `X=cd; $X sub; git checkout -- + dirty.py; X=somethingelse` (reusing a variable name for a later, + unrelated purpose, the same ordinary idiom round 19's own finding + used) resolved to a CONFIDENT, WRONG `checkout_restore_paths` claim + -- `$X` genuinely was `cd` at its actual point of use one statement + earlier -- and was wrongly allowed outright through the real wrapper. + The deny here is classifier-level (a token-shape fact, no live git + call), so no such file needs to actually exist for this regression + pin.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("X=cd; $X sub; git checkout -- dirty.py; X=somethingelse", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + +def test_restore_denied_when_a_dynamic_pushd_relocator_is_reassigned_after_use(tmp_path: Path) -> None: + """Companion to the `cd` pin above, for `pushd` and `git restore` -- + the round-20 finding was confirmed live for all three + `_CWD_RELOCATING_COMMANDS` members and both subcommands.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("X=pushd; $X sub; git restore dirty.py; X=somethingelse", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index d3e62a23..ba871c7b 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -1727,7 +1727,7 @@ def test_assigned_raw_values_biased_toward_stays_on_literal_once_assigned() -> N `TOOL=git; ...; TOOL=npm` must still resolve `TOOL` to `git` here, unlike `_assigned_raw_values`'s own plain last-occurrence-wins collapse.""" - assert checker._assigned_raw_values_biased_toward(["TOOL=git", "TOOL=npm"], "git") == {"TOOL": "git"} + assert checker._assigned_raw_values_biased_toward(["TOOL=git", "TOOL=npm"], frozenset({"git"})) == {"TOOL": "git"} def test_assigned_raw_values_biased_toward_locks_on_regardless_of_order() -> None: @@ -1735,7 +1735,7 @@ def test_assigned_raw_values_biased_toward_locks_on_regardless_of_order() -> Non reassignment of the same name and the end result is the same -- this function does not attempt real execution-order tracking, only a bounded "was LITERAL ever assigned to this name" bias.""" - assert checker._assigned_raw_values_biased_toward(["TOOL=npm", "TOOL=git"], "git") == {"TOOL": "git"} + assert checker._assigned_raw_values_biased_toward(["TOOL=npm", "TOOL=git"], frozenset({"git"})) == {"TOOL": "git"} def test_assigned_raw_values_biased_toward_falls_back_to_last_assignment_when_literal_never_seen() -> None: @@ -1743,7 +1743,7 @@ def test_assigned_raw_values_biased_toward_falls_back_to_last_assignment_when_li exactly as `_assigned_raw_values`'s own plain last-occurrence-wins collapse would -- this function only ever WIDENS toward the literal, never changes behavior for a name that was never a candidate.""" - assert checker._assigned_raw_values_biased_toward(["TOOL=npm", "TOOL=yarn"], "git") == {"TOOL": "yarn"} + assert checker._assigned_raw_values_biased_toward(["TOOL=npm", "TOOL=yarn"], frozenset({"git"})) == {"TOOL": "yarn"} @_PROPERTIES @@ -1756,7 +1756,9 @@ def test_assigned_raw_values_biased_toward_matches_plain_collapse_when_never_rea with the plain, order-blind `_assigned_raw_values`.""" assume(decoy_value.lower() != "git") tokens = [f"{name}={decoy_value}", *tail] - assert checker._assigned_raw_values_biased_toward(tokens, "git") == checker._assigned_raw_values(tokens) + assert checker._assigned_raw_values_biased_toward(tokens, frozenset({"git"})) == checker._assigned_raw_values( + tokens + ) def test_find_git_checkout_restore_recognizes_a_git_biased_reassigned_token() -> None: @@ -1817,6 +1819,69 @@ def test_classify_extracts_checkout_paths_when_git_token_is_reassigned_after_a_c assert verdict.checkout_restore_paths == ("dirty.py",) +def test_assigned_raw_values_biased_toward_accepts_several_interchangeable_literals() -> None: + """Model-based, regression pin for the real bypass found live by Step + 8 independent review, twentieth round (issue #1375): LITERALS is a + SET, not a single string, so `cd`, `pushd`, and `popd` -- three + different literals that all answer the same "was the working tree + possibly relocated" question -- are all sticky against a later + reassignment, not just one of them.""" + assert checker._assigned_raw_values_biased_toward(["X=cd", "X=elsewhere"], checker._CWD_RELOCATING_COMMANDS) == { + "X": "cd" + } + assert checker._assigned_raw_values_biased_toward(["X=pushd", "X=elsewhere"], checker._CWD_RELOCATING_COMMANDS) == { + "X": "pushd" + } + + +def test_rule_git_checkout_restore_recognizes_a_cd_biased_reassigned_relocator() -> None: + """Model-based, regression pin for the real bypass found live by Step + 8 independent review, twentieth round (issue #1375): `_rule_git_ + checkout_restore`'s own dynamic-cd-relocation check was fed only the + ordinary, order-blind RAW_ASSIGNED, the IDENTICAL gap round 19 closed + for the sibling git-token-recognition consumer in the same function, + just left open here -- the ordinary reading declines (X's own + collapsed value is "elsewhere", not a relocator), but the + RAW_ASSIGNED_CD_BIASED reading recognizes `cd` was assigned to X at + some point, so the earlier relocation is still flagged and the + checkout is denied rather than confidently, wrongly resolved.""" + segments = [["$X", "sub"], ["git", "checkout", "--", "dirty.py"]] + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "elsewhere"}, {}, {"X": "cd"}) + assert reason is not None + assert "cd" in reason or "pushd" in reason or "popd" in reason + assert resolved == () + + +def test_classify_denies_checkout_when_a_cd_token_is_reassigned_after_use() -> None: + """End-to-end regression pin for the round-20 finding at the + `classify()` level. Confirmed live before this fix: `X=cd; $X sub; + git checkout -- dirty.py; X=somethingelse` resolved to `deny=False` + with a CONFIDENT, WRONG `checkout_restore_paths` claim, even though + `$X` genuinely was `cd` at its actual point of use one statement + earlier.""" + verdict = checker.classify("X=cd; $X sub; git checkout -- dirty.py; X=somethingelse") + assert verdict.deny is True + + +def test_classify_denies_checkout_when_a_pushd_token_is_reassigned_after_use() -> None: + """Companion to the `cd` pin above, for `pushd` -- the round-20 + finding was confirmed live for all three `_CWD_RELOCATING_COMMANDS` + members.""" + verdict = checker.classify("X=pushd; $X sub; git checkout -- dirty.py; X=somethingelse") + assert verdict.deny is True + + +def test_classify_does_not_flag_cd_relocation_for_an_unrelated_reassigned_tool() -> None: + """No false positive: a variable reassigned but NEVER assigned + `cd`/`pushd`/`popd` anywhere in the command must not be flagged as a + possible relocator -- the cd-biased fallback only ever widens + recognition of a name that really was a relocator at some point, + never invents one out of nothing.""" + verdict = checker.classify("X=curl; $X sub; git checkout -- dirty.py; X=wget") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py",) + + @_PROPERTIES @given(name=_IDENTIFIERS, subscript=st.sampled_from(["0", "1", "@", "*", "$i"])) def test_token_is_all_unassigned_refs_recognizes_a_braced_subscript(name: str, subscript: str) -> None: @@ -3618,7 +3683,7 @@ def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_pat command (`git checkout -- a.py; git restore b.py`) accumulate paths from every segment, not just the first.""" segments = [["git", "checkout", "--", *command_paths], ["git", "restore", *command_paths]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) assert reason is None assert resolved == (*command_paths, *command_paths) @@ -3631,7 +3696,7 @@ def test_rule_git_checkout_restore_denies_when_git_dir_env_var_assigned() -> Non token-shape fact) rather than letting the live wrapper check the wrong tree.""" segments = [["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}, {}, {}) assert reason is not None assert resolved == () @@ -3642,7 +3707,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_cd() -> Non the wrapper's own fixed `.cwd` unsound for a LATER checkout/restore segment -- denied outright.""" segments = [["cd", "/tmp"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) assert reason is not None assert resolved == () @@ -3652,7 +3717,7 @@ def test_rule_git_checkout_restore_allows_cd_after_the_checkout_segment() -> Non `cd` in an EARLIER segment -- a `cd` AFTER the checkout/restore segment does not retroactively make the already-scanned segment unsound.""" segments = [["git", "checkout", "--", "f.py"], ["cd", "/tmp"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) assert reason is None assert resolved == ("f.py",) @@ -3668,7 +3733,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_pushd_or_po claim that the wrapper's live check then found clean at the wrong `.cwd`, silently allowing a real, uncommitted-change discard.""" segments = [[relocator, "/tmp"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) assert reason is not None assert resolved == () @@ -3693,7 +3758,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_starts_with_a_ `checkout_restore_paths` claim the same way round 9's own fix closed for the literal case.""" segments = [["$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}) assert reason is not None assert resolved == () @@ -3705,7 +3770,7 @@ def test_rule_git_checkout_restore_allows_a_genuinely_vanishing_dynamic_word() - real bash would run whatever token follows as the actual command word instead, and that token is scanned on its own merits.""" segments = [["${NEVERSET}", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) assert reason is None assert resolved == ("f.py",) @@ -3733,9 +3798,23 @@ def test_rule_git_checkout_restore_allows_a_dynamic_word_resolving_to_something_ checkout), purely because `$EDITOR` is dynamic and non-vanishing. `_dynamic_word_may_resolve_to_a_cwd_relocator` must resolve the word's actual candidate value and only flag when it could genuinely - be `cd`/`pushd`/`popd`.""" + be `cd`/`pushd`/`popd`. + + RAW_ASSIGNED_CD_BIASED here mirrors RAW_ASSIGNED exactly (round 20, + issue #1375's own fourth argument) -- in real production use it is + always built from the SAME token stream as RAW_ASSIGNED and so + always carries the SAME entry for a name never assigned `cd`/`pushd`/ + `popd`; an EMPTY dict here would NOT be equivalent (unlike the + git-biased fallback's own fail-toward-"not git" posture, `_dynamic_ + word_may_resolve_to_a_cwd_relocator`'s own posture fails toward + "might be a relocator" on an unresolvable name, so a dict missing + EDITOR's own entry would wrongly flag it once the first, correctly- + resolving `raw_assigned` reading is bypassed by this test's own + construction -- this is a hand-built-test-consistency requirement, + not a live production gap, since production always keeps the two + dicts' own key sets in sync).""" segments = [["$EDITOR", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}, {}, {"EDITOR": "vim"}) assert reason is None assert resolved == ("f.py",) @@ -3814,7 +3893,7 @@ def test_dynamic_word_may_resolve_to_a_cwd_relocator_true_for_a_still_dynamic_ca def test_rule_git_checkout_restore_denies_a_still_dynamic_candidate() -> None: segments = [["${UNSET:-$OTHER}", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}, {}, {}) assert reason is not None assert resolved == () @@ -3849,7 +3928,7 @@ def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_v assigned) resolved to a CONFIDENT, WRONG `checkout_restore_paths` claim -- real bash genuinely runs `cd sub` there.""" segments = [["$NEVERSET", "$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}) assert reason is not None assert resolved == () @@ -3892,7 +3971,7 @@ def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_r Live-verified before this fix: `X=cd; > /dev/null $X sub; git checkout -- dirty.py` resolved to a confident, wrong ALLOW.""" segments = [[">", "/dev/null", "$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}) assert reason is not None assert resolved == () From 8306e6d3cc3693a01ae40385aea8ed8cb39a175f Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 18:42:22 +0000 Subject: [PATCH 26/46] fix(hooks): close two more reassignment-ambiguity bypasses A fresh, independent adversarial review of this PR's current head (round 21) found two more live, real-data-loss bypasses in the same reassignment-ambiguity class rounds 19-20 already closed for the git-token and cd-relocation consumers. Finding 1: round 20's own cd-biased fix was deliberately scoped to the current `_classify_tokens` invocation's own top-level segments only, reasoning that a `cd`/`pushd`/`popd` inside a `$(...)`/array-literal span runs in an isolated subshell and never relocates the OUTER shell's cwd, so no threading through the recursive chain was needed. That subshell reasoning is correct but incomplete: it misses a reassignment straddling the substitution's OWN boundary, where the relocator is used entirely WITHIN the substitution but the ambiguity lives in the OUTER token stream. `X=cd; y=$($X sub; git checkout -- dirty.py); X=somethingelse` silently missed the relocation. Confirmed live end-to-end through the real wrapper against a scratch repo with `sub/dirty.py` genuinely dirty relative to `sub`: wrongly allowed with exit 0, and actually running it afterward silently discarded the uncommitted edit. Closed by threading `raw_assigned_cd_biased` as a fourth outer-scope parameter through the full recursive chain after all, mirroring round 19's own git-biased threading exactly. Finding 2 (more severe and more broadly reachable than either prior finding): `_resolve_path_tokens`'s own dynamic-path-argument resolution is a THIRD consumer of the order-blind `_assigned_raw_values` collapse, with no bias mechanism at all before this fix. `F=dirty.py; git checkout -- $F; F=other.py` -- no command substitution, no cd/pushd/ popd, not even multiple statements beyond the reassignment itself -- resolved `$F` to `"other.py"` (the last assignment in token order) alone, even though `$F` genuinely was `dirty.py` at its actual point of use. Unlike the other two consumers, this does not merely MISS an invocation -- it produces a CONFIDENT, WRONG path claim, so the live wrapper's own `git diff --quiet` check runs against the harmless file while the real, genuinely dirty one is never checked at all. Confirmed live end-to-end: wrongly allowed with exit 0 through the real wrapper, and actually running the command afterward silently discarded the uncommitted edit. Identically reproducible for `git restore` and through a command substitution's own inner content. There is no single fixed literal to bias toward for an arbitrary path (unlike `git` or `cd`/`pushd`/`popd`), so this fix is different in kind: a new `_assigned_raw_value_history(tokens)` records every DISTINCT value ever assigned to a name (not collapsed to one), and `_resolve_path_tokens` widens a bare/braced whole-token reference (`_BARE_OR_BRACED_VAR_REF_RE`) to every one of that name's historical values as its own separate candidate path, rather than picking one -- over-including (an extra, harmless candidate gets checked) rather than ever silently dropping the one that matters. Deliberately narrower than full soundness: a token that fuses a reference with literal text, a default clause, or an indirect reference keeps its existing, un-widened resolution unchanged, the same "handle the common, simple shape narrowly" scoping this module's other reassignment-bias fixes already use. A new `_merge_raw_value_histories` unions (not shadows) a shared name's history across outer/inner scope, since an inner reassignment might only take effect after `$NAME` was already used with an outer value -- the same execution-order-agnostic caution this module already applies elsewhere. Threaded as a fifth outer-scope parameter (alongside the fourth, cd-biased one from Finding 1) through the same recursive chain: `classify()`, `_classify_tokens`, `_rule_command_substitution_content`, `_rule_array_literal_content`, `_rule_git_checkout_restore`, and now also `_git_checkout_paths`/`_git_restore_paths`. One methodology note for this round's own gate verification: the detection-logic property-coverage gate must be run against the uncommitted working tree (`git diff -- '*.py'`, no `HEAD`) while iterating pre-commit, not `..HEAD` -- the latter silently excludes every not-yet-committed line and can report a false "OK" for a brand-new function's own trigger. Caught and corrected during this round's own verification before committing; a genuine gap in `_assigned_raw_value_history`'s own `_ASSIGN_RE.match` call was found this way and closed with a dedicated `@given` property test. Both subcommands, the top-level and command-substitution shapes, and the history-widening/dedup/fused-token-exemption edge cases are covered end to end (`hooks/test_gitapex_check_bash_safety.py`, through the real wrapper) and at the unit/property level (`tests/test_gitapex_ check_bash_safety_properties.py`). Full gate suite green: ruff check/ format, mypy, xenon (CI thresholds), the (correctly-invoked) detection-logic property-coverage gate, and 100% line+branch coverage on hooks/gitapex_check_bash_safety.py. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 302 +++++++++++++++--- hooks/test_gitapex_check_bash_safety.py | 54 ++++ ...st_gitapex_check_bash_safety_properties.py | 256 +++++++++++---- 3 files changed, 514 insertions(+), 98 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 6880a55e..fc2f2fb1 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -851,6 +851,8 @@ def _rule_command_substitution_content( name_to_value: dict[str, str], name_to_raw_value: dict[str, str], name_to_raw_value_git_biased: dict[str, str], + name_to_raw_value_cd_biased: dict[str, str], + name_to_raw_value_history: dict[str, tuple[str, ...]], ) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `$(...)` command-substitution span's OWN inner content through this module's full rule set -- bash genuinely @@ -960,7 +962,12 @@ def _rule_command_substitution_content( elsewhere in the OUTER command -- e.g. `G=git; x=$($G checkout -- dirty.py); G=notgit` -- is still recognized inside the substitution's own inner content; see `_find_git_checkout_restore`'s own docstring - for what this parameter means and the live bypass it closes.""" + for what this parameter means and the live bypass it closes. + NAME_TO_RAW_VALUE_CD_BIASED and NAME_TO_RAW_VALUE_HISTORY (round 21, + issue #1375) are threaded the same way, for the analogous cd/pushd/ + popd-relocation and path-argument reassignment bypasses -- see + `_assigned_raw_values_biased_toward`'s own second CRITICAL-bug + paragraph and `_assigned_raw_value_history`'s own docstring.""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -988,7 +995,14 @@ def _rule_command_substitution_content( # check would only change how often an empty-content recursive # `classify()` call is skipped, never a real verdict. if inner_text.strip(): - inner_verdict = classify(inner_text, name_to_value, name_to_raw_value, name_to_raw_value_git_biased) + inner_verdict = classify( + inner_text, + name_to_value, + name_to_raw_value, + name_to_raw_value_git_biased, + name_to_raw_value_cd_biased, + name_to_raw_value_history, + ) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) if inner_verdict.deny: @@ -1003,7 +1017,12 @@ def _rule_command_substitution_content( inner_tokens = tokens[i + 2 : span_end - 1] if inner_tokens: inner_verdict = _classify_tokens( - inner_tokens, name_to_value, name_to_raw_value, name_to_raw_value_git_biased + inner_tokens, + name_to_value, + name_to_raw_value, + name_to_raw_value_git_biased, + name_to_raw_value_cd_biased, + name_to_raw_value_history, ) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) @@ -1693,17 +1712,35 @@ def _assigned_raw_values_biased_toward(tokens: list[str], literals: frozenset[st check now also tries `_dynamic_word_may_resolve_to_a_cwd_relocator` against a `raw_assigned_cd_biased` reading -- `_assigned_raw_values_ biased_toward(tokens, _CWD_RELOCATING_COMMANDS)` -- as a fallback - when the ordinary reading declines. Scoped to the current - `_classify_tokens` invocation's own top-level segments only (merged - with plain OUTER_RAW, not a THIRD `..._cd_biased` outer-scope - parameter threaded through the whole recursive chain the way round - 19's git-biased fix was): a `cd`/`pushd`/`popd` occurring INSIDE a - `$(...)`/array-literal span runs in an isolated subshell and never - relocates the OUTER shell's own cwd at real bash runtime regardless - of any outer reassignment, so there is no equivalent live bypass to - close by threading further -- unlike a `git` token, whose own - resolved identity is unaffected by which shell (sub- or otherwise) - evaluates it.""" + when the ordinary reading declines. + + Round 20's own original version scoped RAW_ASSIGNED_CD_BIASED to the + current `_classify_tokens` invocation's own top-level segments only + (merged with plain OUTER_RAW, not threaded as a fourth outer-scope + parameter through the whole recursive chain the way round 19's own + git-biased dict was), reasoning that a `cd`/`pushd`/`popd` occurring + INSIDE a `$(...)`/array-literal span runs in an isolated subshell and + never relocates the OUTER shell's own cwd, so no equivalent bypass + existed to close by threading further. CRITICAL bug found by + independent adversarial review (round 21, issue #1375), independently + reproduced live: that subshell reasoning is correct as far as it + goes, but incomplete -- it does not cover a reassignment straddling + the substitution's OWN boundary, where the ambiguity lives in the + OUTER token stream, not inside the subshell. `X=cd; y=$($X sub; git + checkout -- dirty.py); X=somethingelse` (the relocator `$X` used + entirely WITHIN the substitution, no relocation crossing the subshell + boundary at all) still silently missed the relocation, since the + scoped-down RAW_ASSIGNED_CD_BIASED never saw the OUTER `X= + somethingelse` reassignment that poisoned the recursive call's own + merged OUTER_RAW. Confirmed live end-to-end through the real wrapper + against a scratch repo with `sub/dirty.py` genuinely dirty relative + to `sub`: this command wrongly allowed with exit 0, and actually + running it afterward silently discarded the uncommitted edit; the + identical command with no trailing reassignment correctly denies. + Closed by threading RAW_ASSIGNED_CD_BIASED as a fourth outer-scope + parameter through the full recursive chain after all, mirroring round + 19's own git-biased threading exactly -- see `_classify_tokens`'s own + docstring.""" values: dict[str, str] = {} for token in tokens: if _is_dynamic(token): @@ -1718,6 +1755,108 @@ def _assigned_raw_values_biased_toward(tokens: list[str], literals: frozenset[st return values +# Matches a token that is EXACTLY one bare `$NAME` or braced `${NAME}` +# reference and nothing else -- no surrounding literal text, no default +# clause, no indirect `${!NAME}` reference. Used only by `_resolve_path_ +# tokens`'s own history-widening (see `_assigned_raw_value_history`'s own +# docstring): for this narrow, unambiguous whole-token shape, the token's +# real bash value is exactly NAME's own raw value, with no risk of the +# unbraced-prefix ambiguity `_substitute_var_refs_candidates` itself +# exists to handle for a token containing MORE than just the reference. +_BARE_OR_BRACED_VAR_REF_RE = re.compile(r"^\$([A-Za-z_][A-Za-z0-9_]*)$|^\$\{([A-Za-z_][A-Za-z0-9_]*)\}$") + + +def _assigned_raw_value_history(tokens: list[str]) -> dict[str, tuple[str, ...]]: + """Like `_assigned_raw_values`, but maps each assigned name to the + TUPLE of every DISTINCT raw value assigned to it anywhere in TOKENS + (first-seen order, deduplicated) -- not collapsed to the single + last-occurrence-wins value `_assigned_raw_values` itself returns. + + CRITICAL bug found by independent adversarial review (round 21, issue + #1375) and independently reproduced live: `_resolve_path_tokens`'s + own dynamic-path-argument resolution is a THIRD consumer of the + order-blind `_assigned_raw_values` collapse -- a THIRD instance of + the identical reassignment-ambiguity class rounds 19 and 20 already + closed for the git-token and cd-relocation consumers in this same + feature, just left open here, and the most severely reachable of the + three: no command substitution, no cd/pushd/popd, not even multiple + statements are required -- `F=dirty.py; git checkout -- $F; + F=other.py` resolves `$F` to `"other.py"` (the LAST assignment in + token order), even though `$F` genuinely was `dirty.py` at its actual + point of use. Unlike the other two consumers, this one does not + merely MISS a real invocation -- it produces a CONFIDENT, WRONG path + claim: `checkout_restore_paths=('other.py',)` instead of `('dirty.py', + )`, so the live wrapper's own `git diff --quiet` check runs against + the WRONG, harmless file while the REAL, genuinely dirty `dirty.py` + is never checked at all. Confirmed live end-to-end through the real + wrapper against a scratch repo with a genuinely dirty, tracked + `dirty.py` and a clean `other.py`: this command wrongly allowed with + exit 0, and actually running it afterward silently discarded the + uncommitted edit to `dirty.py`. Identically reproducible for `git + restore $F`, and reachable through a command substitution's own + inner content the same way rounds 19/20's own findings were (see + `_classify_tokens`'s own docstring for the outer-scope threading this + needs). + + There is no single fixed literal to bias toward here (unlike `git` or + `cd`/`pushd`/`popd` -- an arbitrary path has no small, enumerable + target set), so the fix this history dict feeds is different in kind + from `_assigned_raw_values_biased_toward`'s own single-reading bias: + `_resolve_path_tokens` extracts EVERY distinct historical value for a + name referenced by a bare/braced whole-token reference (see + `_BARE_OR_BRACED_VAR_REF_RE`) as its own SEPARATE candidate path, + rather than picking one. This over-includes (an extra, harmless + candidate path gets checked for dirtiness) rather than ever silently + dropping the one that matters -- the same safe direction every other + ambiguity in this module resolves toward. Deliberately narrower than + full soundness: a token that FUSES a reference with literal text, a + default-value clause, or an indirect `${!NAME}` reference is NOT + widened by this mechanism and keeps whatever single reading + `_substitute_var_refs_candidates` already gives it -- the same + "handle the common, simple bare-reference shape narrowly, don't + attempt full generality" scoping this module's own reassignment-bias + fixes already use elsewhere.""" + history: dict[str, list[str]] = {} + for token in tokens: + if _is_dynamic(token): + continue + match = _ASSIGN_RE.match(token) + if not match: + continue + name, value = match.group(1), match.group(2) + values = history.setdefault(name, []) + if value not in values: + values.append(value) + return {name: tuple(values) for name, values in history.items()} + + +def _merge_raw_value_histories( + outer: dict[str, tuple[str, ...]], inner: dict[str, tuple[str, ...]] +) -> dict[str, tuple[str, ...]]: + """Union OUTER's and INNER's own historical-value tuples per name, + deduplicated, rather than letting INNER's own entry for a shared name + silently replace OUTER's (the plain `{**outer, **inner}` shadowing + convention every other scope dict in this module uses). A name + reassigned INSIDE a `$(...)`/array-literal span's own recursive scope + does shadow the outer value for every OTHER purpose in this module + (an inner reassignment genuinely does take effect for the rest of + that subshell's own execution) -- but for HISTORY specifically, + dropping OUTER's own candidates would silently lose a value `$NAME` + might still resolve to at real bash runtime if it is referenced + BEFORE the inner reassignment takes effect, which this module's own + static analysis (no execution-order tracking, see `_assigned_raw_ + values_biased_toward`'s own docstring) cannot rule out -- so both + scopes' own candidates are kept.""" + merged: dict[str, tuple[str, ...]] = {} + for name in {*outer, *inner}: + seen: list[str] = [] + for value in (*outer.get(name, ()), *inner.get(name, ())): + if value not in seen: + seen.append(value) + merged[name] = tuple(seen) + return merged + + def _strip_leading_assignments(seg: list[str]) -> list[str]: """Bash's own simple-command grammar lets zero or more `NAME=value` environment-assignment tokens precede the actual command word (`X=foo @@ -2396,6 +2535,8 @@ def _rule_array_literal_content( name_to_value: dict[str, str], name_to_raw_value: dict[str, str], name_to_raw_value_git_biased: dict[str, str], + name_to_raw_value_cd_biased: dict[str, str], + name_to_raw_value_history: dict[str, tuple[str, ...]], ) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `NAME=(...)` array-literal span's OWN inner content through this module's full rule set -- bash genuinely @@ -2505,7 +2646,10 @@ def _rule_array_literal_content( VALUE/NAME_TO_RAW_VALUE, mirroring `_rule_command_substitution_ content`'s own identical parameter exactly -- see `_find_git_ checkout_restore`'s own docstring for what it means and the live - bypass it closes.""" + bypass it closes. NAME_TO_RAW_VALUE_CD_BIASED and NAME_TO_RAW_VALUE_ + HISTORY (round 21, issue #1375) are threaded the same way -- see + `_assigned_raw_values_biased_toward`'s own second CRITICAL-bug + paragraph and `_assigned_raw_value_history`'s own docstring.""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -2523,7 +2667,12 @@ def _rule_array_literal_content( readings.append((collapsed, " once its own leading unassigned reference(s) word-split away")) for reading, suffix in readings: reading_verdict = _classify_tokens( - reading, name_to_value, name_to_raw_value, name_to_raw_value_git_biased + reading, + name_to_value, + name_to_raw_value, + name_to_raw_value_git_biased, + name_to_raw_value_cd_biased, + name_to_raw_value_history, ) is_git_push = is_git_push or reading_verdict.is_git_push checkout_restore_paths.extend(reading_verdict.checkout_restore_paths) @@ -3392,7 +3541,9 @@ def _is_git_push_segment(seg: list[str], name_to_raw_value: dict[str, str]) -> b _CHECKOUT_BRANCH_CREATION_FLAGS = {"-b", "-B", "--orphan"} -def _resolve_path_tokens(tokens: list[str], name_to_raw_value: dict[str, str]) -> tuple[str | None, tuple[str, ...]]: +def _resolve_path_tokens( + tokens: list[str], name_to_raw_value: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]] +) -> tuple[str | None, tuple[str, ...]]: """Resolve every token in TOKENS to one or more literal path candidates for a `git checkout`/`git restore` invocation. A literal token is used as-is. A dynamic token is resolved via this module's @@ -3423,7 +3574,17 @@ def _resolve_path_tokens(tokens: list[str], name_to_raw_value: dict[str, str]) - silently pass the literal string `${paths[@]}` through as a "resolved" candidate instead of being recognized as still-unresolved. Confirmed live during this function's own development (before this - check was added).""" + check was added). + + NAME_TO_RAW_VALUE_HISTORY (round 21, issue #1375) widens a BARE or + braced whole-token reference (`_BARE_OR_BRACED_VAR_REF_RE`) to every + DISTINCT value ever assigned to that name, not just the single, + possibly-stale one NAME_TO_RAW_VALUE's own order-blind collapse + gives -- see `_assigned_raw_value_history`'s own docstring for the + live bypass this closes. A token that is not exactly a bare/braced + reference (fused with literal text, a default clause, or an indirect + reference) keeps the ordinary, un-widened CANDIDATES resolution + below unchanged.""" paths: list[str] = [] for tok in tokens: if not _is_dynamic(tok): @@ -3437,12 +3598,21 @@ def _resolve_path_tokens(tokens: list[str], name_to_raw_value: dict[str, str]) - "working tree, so this is denied outright", (), ) + bare_match = _BARE_OR_BRACED_VAR_REF_RE.fullmatch(tok) + history = name_to_raw_value_history.get(bare_match.group(1) or bare_match.group(2)) if bare_match else None + if history: + for value in history: + if value not in paths: + paths.append(value) + continue paths.extend(candidates) return None, tuple(paths) def _git_checkout_paths( - tokens_after: list[str], name_to_raw_value: dict[str, str] + tokens_after: list[str], + name_to_raw_value: dict[str, str], + name_to_raw_value_history: dict[str, tuple[str, ...]], ) -> tuple[str | None, tuple[str, ...]]: """checkout_restore_paths for a `git checkout` invocation, TOKENS_AFTER being every segment token following the literal `checkout` word. @@ -3563,17 +3733,19 @@ def _git_checkout_paths( "could append paths at runtime this classifier cannot see, so this is denied outright", (), ) - return _resolve_path_tokens(after, name_to_raw_value) + return _resolve_path_tokens(after, name_to_raw_value, name_to_raw_value_history) positionals = [t for t in tokens_after if not t.startswith("-")] if len(positionals) >= 2: - return _resolve_path_tokens(positionals, name_to_raw_value) + return _resolve_path_tokens(positionals, name_to_raw_value, name_to_raw_value_history) if len(positionals) == 1 and not _is_dynamic(positionals[0]) and positionals[0] in (".", ".."): - return _resolve_path_tokens(positionals, name_to_raw_value) + return _resolve_path_tokens(positionals, name_to_raw_value, name_to_raw_value_history) return None, () def _git_restore_paths( - tokens_after: list[str], name_to_raw_value: dict[str, str] + tokens_after: list[str], + name_to_raw_value: dict[str, str], + name_to_raw_value_history: dict[str, tuple[str, ...]], ) -> tuple[str | None, tuple[str, ...]]: """checkout_restore_paths for a `git restore` invocation, TOKENS_AFTER being every segment token following the literal `restore` word. @@ -3667,7 +3839,7 @@ def _git_restore_paths( i += 1 if saw_staged and not saw_worktree: return None, () - return _resolve_path_tokens(path_tokens, name_to_raw_value) + return _resolve_path_tokens(path_tokens, name_to_raw_value, name_to_raw_value_history) # A whole token that is EXACTLY one `${NAME-}`/`${NAME:-}` (empty default @@ -4233,6 +4405,7 @@ def _rule_git_checkout_restore( raw_assigned: dict[str, str], raw_assigned_git_biased: dict[str, str], raw_assigned_cd_biased: dict[str, str], + raw_assigned_history: dict[str, tuple[str, ...]], ) -> tuple[str | None, tuple[str, ...]]: """Extract every `checkout_restore_paths` candidate across every segment of one command, denying outright on any segment where this @@ -4322,7 +4495,10 @@ def _rule_git_checkout_restore( issue #1375). RAW_ASSIGNED_CD_BIASED feeds a SECOND fallback, for the cd/pushd/popd-relocation check just below (round 20, issue #1375) -- see `_assigned_raw_values_biased_toward`'s own second CRITICAL-bug - paragraph for the live bypass this one closes.""" + paragraph for the live bypass this one closes. RAW_ASSIGNED_HISTORY + is passed straight through into `_git_checkout_paths`/`_git_restore_ + paths` (round 21, issue #1375) -- see `_assigned_raw_value_history`'s + own docstring for what it means and the live bypass it closes.""" saw_cd = False all_paths: list[str] = [] for seg in segments: @@ -4350,9 +4526,9 @@ def _rule_git_checkout_restore( (), ) if subcommand == "checkout": - deny_reason, paths = _git_checkout_paths(tokens_after, raw_assigned) + deny_reason, paths = _git_checkout_paths(tokens_after, raw_assigned, raw_assigned_history) else: - deny_reason, paths = _git_restore_paths(tokens_after, raw_assigned) + deny_reason, paths = _git_restore_paths(tokens_after, raw_assigned, raw_assigned_history) if deny_reason: return deny_reason, () all_paths.extend(paths) @@ -4559,6 +4735,8 @@ def classify( outer_name_to_value: dict[str, str] | None = None, outer_name_to_raw_value: dict[str, str] | None = None, outer_name_to_raw_value_git_biased: dict[str, str] | None = None, + outer_name_to_raw_value_cd_biased: dict[str, str] | None = None, + outer_name_to_raw_value_history: dict[str, tuple[str, ...]] | None = None, ) -> Verdict: """Classify one Bash tool_input.command string. Fails closed (deny) on anything shlex cannot tokenize -- an unparseable command is exactly the @@ -4575,15 +4753,24 @@ def classify( `$(...)` span's own inner content can resolve a variable assigned OUTSIDE the span against the same shell scope real bash would use. - OUTER_NAME_TO_RAW_VALUE_GIT_BIASED (round 19, issue #1375) is the - same recursive call's own analogous third argument -- see `_classify_ - tokens`'s own docstring and `_find_git_checkout_restore`'s own - docstring for what it means and the live bypass it closes.""" + OUTER_NAME_TO_RAW_VALUE_GIT_BIASED (round 19, issue #1375), + OUTER_NAME_TO_RAW_VALUE_CD_BIASED and OUTER_NAME_TO_RAW_VALUE_HISTORY + (round 21, issue #1375) are the same recursive call's own analogous + further arguments -- see `_classify_tokens`'s own docstring and + `_find_git_checkout_restore`'s/`_assigned_raw_value_history`'s own + docstrings for what each means and the live bypass each closes.""" try: tokens = tokenize(command) except TokenizeError as error: return Verdict(True, f"the command could not be parsed as shell syntax ({error}). Failing closed", False) - return _classify_tokens(tokens, outer_name_to_value, outer_name_to_raw_value, outer_name_to_raw_value_git_biased) + return _classify_tokens( + tokens, + outer_name_to_value, + outer_name_to_raw_value, + outer_name_to_raw_value_git_biased, + outer_name_to_raw_value_cd_biased, + outer_name_to_raw_value_history, + ) def _classify_tokens( @@ -4591,6 +4778,8 @@ def _classify_tokens( outer_name_to_value: dict[str, str] | None = None, outer_name_to_raw_value: dict[str, str] | None = None, outer_name_to_raw_value_git_biased: dict[str, str] | None = None, + outer_name_to_raw_value_cd_biased: dict[str, str] | None = None, + outer_name_to_raw_value_history: dict[str, tuple[str, ...]] | None = None, ) -> Verdict: """The token-level core of `classify` -- split out so `_rule_command_ substitution_content` can recurse into a `$(...)` span's own inner @@ -4631,25 +4820,60 @@ def _classify_tokens( something else LATER in the same command (in an outer scope, inside a `$(...)`/array-literal span, or both) still resolves to `git` for checkout/restore recognition specifically. See `_find_git_checkout_ - restore`'s own docstring for the live bypass this closes.""" + restore`'s own docstring for the live bypass this closes. + + OUTER_NAME_TO_RAW_VALUE_CD_BIASED and OUTER_NAME_TO_RAW_VALUE_HISTORY + (round 21, issue #1375) are two further, parallel outer-scope + arguments, merged the identical way -- `_assigned_raw_values_biased_ + toward(tokens, _CWD_RELOCATING_COMMANDS)` for the former, `_merge_ + raw_value_histories(outer, _assigned_raw_value_history(tokens))` (a + UNION merge, not the plain `{**outer, **inner}` shadowing every other + dict here uses -- see that function's own docstring for why) for the + latter -- and threaded the same way into the same two recursive + calls plus `_rule_git_checkout_restore`'s own fourth and fifth + arguments. CD_BIASED closes the cd/pushd/popd-relocation analogue of + the git-token bypass just above; HISTORY closes a live path-argument- + resolution bypass round 20's own scoped-down (non-recursively- + threaded) cd-biased fix did NOT cover -- see `_assigned_raw_values_ + biased_toward`'s own second CRITICAL-bug paragraph and `_assigned_ + raw_value_history`'s own docstring for both bypasses.""" outer_literals = outer_name_to_value or {} outer_raw = outer_name_to_raw_value or {} outer_raw_git_biased = outer_name_to_raw_value_git_biased or {} + outer_raw_cd_biased = outer_name_to_raw_value_cd_biased or {} + outer_raw_history = outer_name_to_raw_value_history or {} merged_name_to_value = {**outer_literals, **_assigned_literals(tokens)} merged_name_to_raw_value = {**outer_raw, **_assigned_raw_values(tokens)} merged_name_to_raw_value_git_biased = { **outer_raw_git_biased, **_assigned_raw_values_biased_toward(tokens, frozenset({"git"})), } + merged_name_to_raw_value_cd_biased = { + **outer_raw_cd_biased, + **_assigned_raw_values_biased_toward(tokens, _CWD_RELOCATING_COMMANDS), + } + merged_name_to_raw_value_history = _merge_raw_value_histories( + outer_raw_history, _assigned_raw_value_history(tokens) + ) content_reason, content_is_git_push, content_checkout_restore_paths = _rule_command_substitution_content( - tokens, merged_name_to_value, merged_name_to_raw_value, merged_name_to_raw_value_git_biased + tokens, + merged_name_to_value, + merged_name_to_raw_value, + merged_name_to_raw_value_git_biased, + merged_name_to_raw_value_cd_biased, + merged_name_to_raw_value_history, ) if content_reason: return Verdict(True, content_reason, content_is_git_push, content_checkout_restore_paths) array_content_reason, array_content_is_git_push, array_content_checkout_restore_paths = _rule_array_literal_content( - tokens, merged_name_to_value, merged_name_to_raw_value, merged_name_to_raw_value_git_biased + tokens, + merged_name_to_value, + merged_name_to_raw_value, + merged_name_to_raw_value_git_biased, + merged_name_to_raw_value_cd_biased, + merged_name_to_raw_value_history, ) is_git_push = content_is_git_push or array_content_is_git_push checkout_restore_paths = content_checkout_restore_paths + array_content_checkout_restore_paths @@ -4664,7 +4888,11 @@ def _classify_tokens( **outer_raw_git_biased, **_assigned_raw_values_biased_toward(tokens, frozenset({"git"})), } - raw_assigned_cd_biased = {**outer_raw, **_assigned_raw_values_biased_toward(tokens, _CWD_RELOCATING_COMMANDS)} + raw_assigned_cd_biased = { + **outer_raw_cd_biased, + **_assigned_raw_values_biased_toward(tokens, _CWD_RELOCATING_COMMANDS), + } + raw_assigned_history = _merge_raw_value_histories(outer_raw_history, _assigned_raw_value_history(tokens)) lowered_command = " ".join(tokens).lower() is_git_push = is_git_push or any(_is_git_push_segment(seg, raw_assigned) for seg in segments) @@ -4697,7 +4925,7 @@ def _classify_tokens( ) own_checkout_restore_hit, own_checkout_restore_paths = _rule_git_checkout_restore( - segments, raw_assigned, raw_assigned_git_biased, raw_assigned_cd_biased + segments, raw_assigned, raw_assigned_git_biased, raw_assigned_cd_biased, raw_assigned_history ) checkout_restore_paths = checkout_restore_paths + own_checkout_restore_paths if own_checkout_restore_hit: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index b0aadc98..b85d8f99 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1991,6 +1991,60 @@ def test_restore_denied_when_a_dynamic_pushd_relocator_is_reassigned_after_use(t assert "working tree is at risk" in payload["systemMessage"] +def test_checkout_denied_when_a_dynamic_cd_relocator_is_reassigned_across_a_command_substitution( + tmp_path: Path, +) -> None: + """CRITICAL bypass regression pin (round-21 independent review, issue + #1375). Round 20's own cd-biased fix was scoped to the current + `_classify_tokens` invocation's own top-level segments only, which + missed a reassignment straddling a command substitution's OWN + boundary -- the relocator `$X` is used entirely WITHIN the + substitution, but the ambiguity lives in the OUTER token stream. + Live-verified before this fix: `X=cd; y=$($X sub; git checkout -- + dirty.py); X=somethingelse` was wrongly allowed outright through the + real wrapper. The deny here is classifier-level (a token-shape fact, + no live git call), so no such file needs to actually exist for this + regression pin.""" + repo_dir = tmp_path / "repo" + _init_repo_with_committed_file(repo_dir) + result = run("X=cd; y=$($X sub; git checkout -- dirty.py); X=somethingelse", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + payload = json.loads(result.stderr) + assert "working tree is at risk" in payload["systemMessage"] + + +def test_checkout_denied_when_a_path_argument_is_reassigned_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-21 independent review, issue + #1375). `_resolve_path_tokens`'s own dynamic-path-argument resolution + is a THIRD consumer of the order-blind `_assigned_raw_values` + collapse, with no bias mechanism at all before this fix -- the most + severely reachable of the three reassignment-ambiguity bugs found in + this feature (rounds 19-21): no command substitution, no cd/pushd/ + popd, not even multiple statements beyond the reassignment itself are + required. Live-verified before this fix: `F=dirty.py; git checkout -- + $F; F=other.py` resolved `checkout_restore_paths` to `('other.py',)` + alone -- a CONFIDENT, WRONG claim, since `$F` genuinely was + `dirty.py` at its actual point of use -- so the real wrapper's own + live `git diff --quiet` check ran against the harmless `other.py` + and never checked the genuinely dirty `dirty.py` at all, wrongly + allowing the command outright.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("F=dirty.py; git checkout -- $F; F=other.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_restore_denied_when_a_path_argument_is_reassigned_after_use(tmp_path: Path) -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-21 finding was confirmed live for both subcommands.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("F=dirty.py; git restore $F; F=other.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index ba871c7b..fa1ca82b 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -1358,7 +1358,7 @@ def test_rule_command_substitution_content_detects_an_embedded_install(tool: str a punctuation character shlex breaks a word at, so an assignment's `NAME=` prefix stays fused onto the leading `$` in the same token.""" tokens = ["x=$", "(", tool, "install", "evil-pkg", ")"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) assert reason is not None @@ -1374,7 +1374,7 @@ def test_rule_command_substitution_content_allows_harmless_inner_content(value: silently dropping a non-denying inner `is_git_push=True` signal (see the function's own docstring).""" tokens = ["echo", "$", "(", "date", value, ")"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) # --- Issue #1326 Stage 1, fifteenth round: bash's own leading-assignment ---- @@ -1493,7 +1493,7 @@ def test_rule_array_literal_content_detects_a_denied_pair_regardless_of_a_leadin `Y=1; A=(uv install $Y); "${A[@]}"` was wrongly ALLOWED before this function existed.""" tokens = ["dummy=", "(", f"${first}", "uv", "install", f"${second}", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) assert reason is not None @@ -1512,7 +1512,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_bare_ref(unse fused with other text (not a bare whole-token reference), must NOT be collapsed -- that shape does not word-split away to nothing.""" tokens = ["dummy=", "(", f"${unset_name}", verb_a, "install", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) assert reason is not None @@ -1532,13 +1532,13 @@ def test_rule_array_literal_content_allows_harmless_content() -> None: denied pattern, with or without a leading unassigned reference, stays allowed.""" tokens = ["dummy=", "(", "$NEVERSET", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_no_span_present() -> None: """Robustness: a token stream with no array-literal span at all (e.g. an ordinary command) returns cleanly, never a crash.""" - assert checker._rule_array_literal_content(["echo", "hi"], {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(["echo", "hi"], {}, {}, {}, {}, {}) == (None, False, ()) def test_strip_leading_unassigned_bare_refs_stops_at_a_fused_token() -> None: @@ -1564,7 +1564,7 @@ def test_rule_array_literal_content_empty_array_is_harmless() -> None: """No false positive / no crash: an empty array literal `NAME=()` has no inner content to recursively classify at all.""" tokens = ["dummy=", "(", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leading_unassigned_ref() -> None: @@ -1573,7 +1573,7 @@ def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leadin `_strip_leading_unassigned_bare_refs` to strip -- the collapsed reading equals the as-is one, so only one classification is needed.""" tokens = ["dummy=", "(", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> None: @@ -1590,7 +1590,7 @@ def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> No real, with a dynamic verb argument right after it -- exactly B2's own watched shape.""" tokens = ["dummy=", "(", "$NEVERSET", "uv", "$VERB", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) assert reason is not None assert "unassigned reference" in reason @@ -1632,7 +1632,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_braced_bare_r this round, silently degrading the collapsed reading to a no-op for this shape.""" tokens = ["dummy=", "(", f"${{{unset_name}}}", verb_a, "install", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) assert reason is not None @@ -1647,7 +1647,7 @@ def test_rule_array_literal_content_detects_an_outer_scope_resolved_pair() -> No recursive `_classify_tokens` call.""" tokens = ["dummy=", "(", "$G", "$P", "$M", ")"] outer = {"G": "gh", "P": "pr", "M": "merge"} - reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer, outer) + reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer, outer, {}, {}) assert reason is not None @@ -1683,7 +1683,7 @@ def test_rule_command_substitution_content_detects_an_outer_scope_resolved_check for the array-literal span.""" tokens = ["x=$", "(", "$G", "checkout", "--", "dirty.py", ")"] outer = {"G": "git"} - reason, _, checkout_restore_paths = checker._rule_command_substitution_content(tokens, outer, outer, outer) + reason, _, checkout_restore_paths = checker._rule_command_substitution_content(tokens, outer, outer, outer, {}, {}) assert reason is None assert checkout_restore_paths == ("dirty.py",) @@ -1846,7 +1846,7 @@ def test_rule_git_checkout_restore_recognizes_a_cd_biased_reassigned_relocator() some point, so the earlier relocation is still flagged and the checkout is denied rather than confidently, wrongly resolved.""" segments = [["$X", "sub"], ["git", "checkout", "--", "dirty.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "elsewhere"}, {}, {"X": "cd"}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "elsewhere"}, {}, {"X": "cd"}, {}) assert reason is not None assert "cd" in reason or "pushd" in reason or "popd" in reason assert resolved == () @@ -1882,6 +1882,140 @@ def test_classify_does_not_flag_cd_relocation_for_an_unrelated_reassigned_tool() assert verdict.checkout_restore_paths == ("dirty.py",) +def test_classify_denies_checkout_when_a_cd_token_is_reassigned_across_a_command_substitution() -> None: + """End-to-end regression pin for the round-21 finding at the + `classify()` level: round 20's own cd-biased fix was scoped to the + current `_classify_tokens` invocation's own top-level segments only, + which missed a reassignment straddling a command substitution's OWN + boundary -- the relocator `$X` is used entirely WITHIN the + substitution, but the ambiguity lives in the OUTER token stream. + Confirmed live before this fix: `X=cd; y=$($X sub; git checkout -- + dirty.py); X=somethingelse` resolved to `deny=False` with a + CONFIDENT, WRONG `checkout_restore_paths` claim.""" + verdict = checker.classify("X=cd; y=$($X sub; git checkout -- dirty.py); X=somethingelse") + assert verdict.deny is True + + +def test_assigned_raw_value_history_records_every_distinct_value() -> None: + """Model-based, regression pin for the real bypass found live by Step + 8 independent review, twenty-first round (issue #1375): unlike + `_assigned_raw_values`'s own last-occurrence-wins collapse, every + DISTINCT value ever assigned to a name is kept, in first-seen order.""" + assert checker._assigned_raw_value_history(["F=dirty.py", "F=other.py"]) == {"F": ("dirty.py", "other.py")} + + +def test_assigned_raw_value_history_deduplicates_an_identical_reassignment() -> None: + """The SAME value assigned twice to the same name contributes only + ONE entry to its history, not a duplicate.""" + assert checker._assigned_raw_value_history(["F=dirty.py", "F=dirty.py"]) == {"F": ("dirty.py",)} + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value1=_VALUES, value2=_VALUES, tail=st.lists(_IDENTIFIERS, max_size=2)) +def test_assigned_raw_value_history_matches_last_assignment_of_assigned_raw_values( + name: str, value1: str, value2: str, tail: list[str] +) -> None: + """Model-based: `_assigned_raw_value_history`'s own last entry for a + name always agrees with `_assigned_raw_values`'s own single, + last-occurrence-wins value for that same name -- the history is a + strict widening (every value `_assigned_raw_values` itself could ever + report, plus every earlier one it silently discarded), never a + disagreement.""" + tokens = [f"{name}={value1}", f"{name}={value2}", *tail] + history = checker._assigned_raw_value_history(tokens) + assert history[name][-1] == checker._assigned_raw_values(tokens)[name] + + +def test_merge_raw_value_histories_unions_rather_than_shadows() -> None: + """Model-based: unlike the plain `{**outer, **inner}` shadowing + convention every other scope dict in this module uses, a name + appearing in BOTH outer and inner histories keeps candidates from + BOTH, not just inner's own -- an inner reassignment could genuinely + take effect only AFTER `$NAME` was already used with an outer value, + which this module's own static analysis cannot rule out (see this + function's own docstring).""" + merged = checker._merge_raw_value_histories({"F": ("dirty.py",)}, {"F": ("other.py",)}) + assert merged == {"F": ("dirty.py", "other.py")} + + +def test_merge_raw_value_histories_deduplicates_a_value_shared_by_both_scopes() -> None: + """A value present in BOTH outer's and inner's own history for the + same name contributes only one entry to the merged result.""" + merged = checker._merge_raw_value_histories({"F": ("dirty.py", "other.py")}, {"F": ("other.py",)}) + assert merged == {"F": ("dirty.py", "other.py")} + + +def test_resolve_path_tokens_widens_a_bare_reference_to_its_full_history() -> None: + """Model-based, regression pin for the real bypass found live by Step + 8 independent review, twenty-first round (issue #1375): + `_resolve_path_tokens`'s own dynamic-path-argument resolution is a + THIRD consumer of the order-blind `_assigned_raw_values` collapse, + with no bias mechanism at all before this fix -- `F=dirty.py; git + checkout -- $F; F=other.py` resolved `$F` to `"other.py"` (the LAST + assignment in token order) alone, even though `$F` genuinely was + `dirty.py` at its actual point of use, so the REAL dirty file was + never checked. A bare/braced whole-token reference to a name with + multiple distinct historical values now extracts ALL of them as + separate candidates, not just the single, possibly-stale one the + collapsed NAME_TO_RAW_VALUE dict gives.""" + reason, resolved = checker._resolve_path_tokens(["$F"], {"F": "other.py"}, {"F": ("dirty.py", "other.py")}) + assert reason is None + assert resolved == ("dirty.py", "other.py") + + +def test_resolve_path_tokens_history_widening_deduplicates_against_existing_paths() -> None: + """A historical value that coincides with a path already extracted + from an earlier, literal token in the same command is not appended + twice.""" + reason, resolved = checker._resolve_path_tokens( + ["dirty.py", "$F"], {"F": "other.py"}, {"F": ("dirty.py", "other.py")} + ) + assert reason is None + assert resolved == ("dirty.py", "other.py") + + +def test_resolve_path_tokens_does_not_widen_a_fused_reference() -> None: + """No false positive: a token that FUSES a reference with literal + text is NOT widened by the history mechanism (`_BARE_OR_BRACED_VAR_ + REF_RE` only matches a WHOLE-token bare/braced reference) -- it keeps + the ordinary, single-candidate resolution unchanged, the same + documented, deliberately narrower-than-full-soundness scoping this + fix uses everywhere else.""" + reason, resolved = checker._resolve_path_tokens(["${F}.py"], {"F": "dirty"}, {"F": ("dirty", "other")}) + assert reason is None + assert resolved == ("dirty.py",) + + +def test_classify_extracts_every_historical_path_when_a_checkout_path_is_reassigned_after_use() -> None: + """End-to-end regression pin for the round-21 finding at the + `classify()` level. Confirmed live before this fix: `F=dirty.py; git + checkout -- $F; F=other.py` resolved to `checkout_restore_paths= + ('other.py',)` -- a CONFIDENT, WRONG claim, since `$F` genuinely was + `dirty.py` at its actual point of use -- instead of including the + real, at-risk path at all.""" + verdict = checker.classify("F=dirty.py; git checkout -- $F; F=other.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py", "other.py") + + +def test_classify_extracts_every_historical_path_when_a_restore_path_is_reassigned_after_use() -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-21 finding was confirmed live for both subcommands.""" + verdict = checker.classify("F=dirty.py; git restore $F; F=other.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py", "other.py") + + +def test_classify_extracts_every_historical_path_behind_a_command_substitution() -> None: + """Companion to the two pins above, for the command-substitution + shape: the SAME reassignment-after-use gap, reached through + `_rule_command_substitution_content`'s own outer-scope threading + (rounds 18-21).""" + verdict = checker.classify("F=dirty.py; x=$(git checkout -- $F); F=other.py") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("dirty.py", "other.py") + + @_PROPERTIES @given(name=_IDENTIFIERS, subscript=st.sampled_from(["0", "1", "@", "*", "$i"])) def test_token_is_all_unassigned_refs_recognizes_a_braced_subscript(name: str, subscript: str) -> None: @@ -1956,7 +2090,7 @@ def test_rule_array_literal_content_detects_a_braced_subscript_decoy() -> None: the subscript decoy blocked it from ever firing until it collapsed away.""" tokens = ["dummy=", "(", "${NEVERSET[0]}", "uv", "$VERB", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) assert reason is not None @@ -1967,7 +2101,7 @@ def test_rule_array_literal_content_detects_a_fused_reference_chain_decoy() -> N before a fused chain of two bare references was recognized as vanishing as a unit.""" tokens = ["dummy=", "(", "$A_UNSET$B_UNSET", "gh", "pr", "merge", "1", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) assert reason is not None @@ -2100,7 +2234,7 @@ def test_rule_command_substitution_content_scans_second_fused_span_in_same_token this test only proves that fix reached end-to-end through `_rule_command_substitution_content`'s own scan loop.""" tokens = ["echo", "$(echo ok)$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) assert reason is not None @@ -2109,20 +2243,20 @@ def test_rule_command_substitution_content_skips_blank_fused_span_then_finds_den skipped without denying by itself, but scanning continues to the next fused span in the same token.""" tokens = ["echo", "$( )$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) assert reason is not None def test_rule_command_substitution_content_both_fused_spans_harmless() -> None: tokens = ["echo", "$(echo ok)$(echo also-ok)"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) def test_rule_command_substitution_content_empty_unquoted_span_skipped() -> None: """An empty, unquoted `$()` substitution has no inner tokens to recurse into -- distinct from the fused/quoted empty-span case above.""" tokens = ["$", "(", ")"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) def test_tokenize_raises_on_unbalanced_quote() -> None: @@ -2846,7 +2980,7 @@ def test_main_allows_a_harmless_command(monkeypatch: pytest.MonkeyPatch, capsys: def test_resolve_path_tokens_returns_literal_tokens_unchanged(paths: list[str]) -> None: """Model-based: every literal (non-dynamic) token is returned as-is, in order, with no deny reason.""" - reason, resolved = checker._resolve_path_tokens(paths, {}) + reason, resolved = checker._resolve_path_tokens(paths, {}, {}) assert reason is None assert resolved == tuple(paths) @@ -2864,7 +2998,7 @@ def test_resolve_path_tokens_resolves_a_braced_reference_case_preserved(name: st against the lowercased map and would have silently mismatched a mixed-case path like `README.md` against `readme.md`.""" mixed_case_value = value.swapcase() - reason, resolved = checker._resolve_path_tokens([f"${{{name}}}"], {name: mixed_case_value}) + reason, resolved = checker._resolve_path_tokens([f"${{{name}}}"], {name: mixed_case_value}, {}) assert reason is None assert resolved == (mixed_case_value,) @@ -2878,7 +3012,7 @@ def test_resolve_path_tokens_denies_an_unresolvable_dynamic_token(name: str) -> `git diff --quiet HEAD -- PATH` exits 0 (clean) for a path that does not exist (issue #1375 Fact 5, confirmed live), which would be fail-open.""" - reason, resolved = checker._resolve_path_tokens([f"${name}"], {}) + reason, resolved = checker._resolve_path_tokens([f"${name}"], {}, {}) assert reason is not None assert resolved == () @@ -2892,7 +3026,7 @@ def test_resolve_path_tokens_denies_an_array_subscript_token() -> None: token's own text UNCHANGED -- silently treating an unexpanded shell construct as though it were already a resolved literal path. Must deny, not pass `${paths[@]}` through as a literal filename.""" - reason, resolved = checker._resolve_path_tokens(["${paths[@]}"], {}) + reason, resolved = checker._resolve_path_tokens(["${paths[@]}"], {}, {}) assert reason is not None assert resolved == () @@ -2902,7 +3036,7 @@ def test_resolve_path_tokens_denies_an_array_subscript_token() -> None: def test_git_checkout_paths_extracts_every_token_after_double_dash(paths: list[str]) -> None: """Model-based, sub-case (a): every token after a literal `--` is a path -- the near-miss's own exact shape (`git checkout -- PATH`).""" - reason, resolved = checker._git_checkout_paths(["--", *paths], {}) + reason, resolved = checker._git_checkout_paths(["--", *paths], {}, {}) assert reason is None assert resolved == tuple(paths) @@ -2912,7 +3046,7 @@ def test_git_checkout_paths_denies_double_dash_with_nothing_following() -> None: harmless no-op in real git by itself, but a downstream pipe/loop could still append paths at runtime this classifier cannot see, and denying a genuine no-op costs nothing.""" - reason, resolved = checker._git_checkout_paths(["--"], {}) + reason, resolved = checker._git_checkout_paths(["--"], {}, {}) assert reason is not None assert resolved == () @@ -2925,7 +3059,7 @@ def test_git_checkout_paths_extracts_two_or_more_positionals_with_no_double_dash `git checkout no-such-ref no-such-file` reports a pathspec error for BOTH arguments, so every position past the first is a pathspec under every resolution real git can take once one exists at all.""" - reason, resolved = checker._git_checkout_paths(paths, {}) + reason, resolved = checker._git_checkout_paths(paths, {}, {}) assert reason is None assert resolved == tuple(paths) @@ -2938,7 +3072,7 @@ def test_git_checkout_paths_treats_a_single_dot_or_dotdot_positional_as_a_path(d (confirmed live: `git check-ref-format --branch .`/`--branch ..` both fail), and `git checkout .` on a dirty tracked file was confirmed live to silently discard the change.""" - reason, resolved = checker._git_checkout_paths([dot], {}) + reason, resolved = checker._git_checkout_paths([dot], {}, {}) assert reason is None assert resolved == (dot,) @@ -2951,7 +3085,7 @@ def test_git_checkout_paths_is_a_non_goal_for_a_single_bare_positional(name: str `git checkout SOMENAME` from a branch/ref name needs a live ref-existence lookup this pure classifier does not perform.""" assume(name not in (".", "..")) - reason, resolved = checker._git_checkout_paths([name], {}) + reason, resolved = checker._git_checkout_paths([name], {}, {}) assert reason is None assert resolved == () @@ -2960,7 +3094,7 @@ def test_git_checkout_paths_allows_a_flag_only_invocation() -> None: """No false positive: `git checkout -b new-branch` has one flag-shaped and one non-flag-shaped token, but the non-flag token is a branch name, not `.`/`..` -- stays the Non-goal, empty paths.""" - reason, resolved = checker._git_checkout_paths(["-b", "new-branch"], {}) + reason, resolved = checker._git_checkout_paths(["-b", "new-branch"], {}, {}) assert reason is None assert resolved == () @@ -2971,7 +3105,7 @@ def test_git_restore_paths_empty_when_staged_without_worktree(staged: str, paths """Model-based: `--staged`/`-S` without `--worktree` never touches the working tree -- empty `checkout_restore_paths`, never live-checked, regardless of what path arguments are also present.""" - reason, resolved = checker._git_restore_paths([staged, *paths], {}) + reason, resolved = checker._git_restore_paths([staged, *paths], {}, {}) assert reason is None assert resolved == () @@ -2983,7 +3117,7 @@ def test_git_restore_paths_checked_when_staged_and_worktree_both_present(worktre --worktree PATH` is a real working-tree-affecting restore despite `--staged` being present -- `saw_worktree=True` must still force the path to be checked.""" - reason, resolved = checker._git_restore_paths(["--staged", worktree, *paths], {}) + reason, resolved = checker._git_restore_paths(["--staged", worktree, *paths], {}, {}) assert reason is None assert resolved == tuple(paths) @@ -2993,7 +3127,7 @@ def test_git_restore_paths_checked_when_staged_and_worktree_both_present(worktre def test_git_restore_paths_checked_with_no_flags_at_all(paths: list[str]) -> None: """Model-based: a bare `git restore PATH` with no flags at all is never staged-only-safe -- always checked.""" - reason, resolved = checker._git_restore_paths(paths, {}) + reason, resolved = checker._git_restore_paths(paths, {}, {}) assert reason is None assert resolved == tuple(paths) @@ -3006,7 +3140,7 @@ def test_git_restore_paths_checked_for_source_short_flag_not_conflated_with_stag (`--staged`, boolean) the way a lower-casing flag scan (like `_is_git_push_segment`'s own) would -- `git restore -s main PATH` stays checked, not wrongly read as staged-only-safe.""" - reason, resolved = checker._git_restore_paths(["-s", ref, *paths], {}) + reason, resolved = checker._git_restore_paths(["-s", ref, *paths], {}, {}) assert reason is None assert resolved == tuple(paths) @@ -3018,7 +3152,7 @@ def test_git_restore_paths_last_occurrence_wins_for_staged(last: str) -> None: --no-staged` ends with `saw_staged=False` (checked), and `--no-staged --staged` ends with `saw_staged=True` (empty, iff no `--worktree`).""" flags = ["--no-staged", "--staged"] if last == "--staged" else ["--staged", "--no-staged"] - reason, resolved = checker._git_restore_paths([*flags, "f.py"], {}) + reason, resolved = checker._git_restore_paths([*flags, "f.py"], {}, {}) assert reason is None if last == "--staged": assert resolved == () @@ -3033,7 +3167,7 @@ def test_git_restore_paths_last_occurrence_wins_for_worktree(paths: list[str]) - `--staged --worktree --no-worktree` ends with `saw_worktree=False`, so the invocation is safe (staged, not worktree) and never live-checked -- exercises the `--no-worktree` branch directly.""" - reason, resolved = checker._git_restore_paths(["--staged", "--worktree", "--no-worktree", *paths], {}) + reason, resolved = checker._git_restore_paths(["--staged", "--worktree", "--no-worktree", *paths], {}, {}) assert reason is None assert resolved == () @@ -3050,7 +3184,7 @@ def test_git_restore_paths_every_boolean_flag_consumes_no_value(flag: str, paths `--ignore-unmerged`, `--ignore-skip-worktree-bits`) is skipped without consuming the token after it as a value -- the following path tokens are still extracted.""" - reason, resolved = checker._git_restore_paths([flag, *paths], {}) + reason, resolved = checker._git_restore_paths([flag, *paths], {}, {}) assert reason is None assert resolved == tuple(paths) @@ -3061,10 +3195,10 @@ def test_git_restore_paths_recurse_submodules_bare_and_fused(value: str, paths: """Model-based: `--recurse-submodules` (bare, consumes nothing) and `--recurse-submodules=VALUE` (fused, self-contained) are both skipped without treating the next token as a value or as part of the flag.""" - reason, resolved = checker._git_restore_paths(["--recurse-submodules", *paths], {}) + reason, resolved = checker._git_restore_paths(["--recurse-submodules", *paths], {}, {}) assert reason is None assert resolved == tuple(paths) - reason, resolved = checker._git_restore_paths([f"--recurse-submodules={value}", *paths], {}) + reason, resolved = checker._git_restore_paths([f"--recurse-submodules={value}", *paths], {}, {}) assert reason is None assert resolved == tuple(paths) @@ -3095,7 +3229,7 @@ def test_git_restore_paths_extracts_every_token_after_double_dash(paths: list[st """`--` disambiguates every remaining token as a pathspec for `git restore`, the identical role it plays for `git checkout` -- must be recognized, not denied as an unrecognized flag.""" - reason, resolved = checker._git_restore_paths(["--", *paths], {}) + reason, resolved = checker._git_restore_paths(["--", *paths], {}, {}) assert reason is None assert resolved == tuple(paths) @@ -3110,7 +3244,7 @@ def test_git_restore_paths_recognizes_fused_value_flags(flag_value: tuple[str, s legitimate git syntax as the separate-token form already recognized -- must not be denied as an unrecognized flag.""" flag, value = flag_value - reason, resolved = checker._git_restore_paths([f"{flag}={value}", *paths], {}) + reason, resolved = checker._git_restore_paths([f"{flag}={value}", *paths], {}, {}) assert reason is None assert resolved == tuple(paths) @@ -3128,7 +3262,7 @@ def test_git_restore_paths_denies_pathspec_from_file(flag: str) -> None: outright rather than silently under-extracting (an empty `checkout_restore_paths` would be exactly issue #1375 Fact 5's own fail-open shape).""" - reason, resolved = checker._git_restore_paths([flag], {}) + reason, resolved = checker._git_restore_paths([flag], {}, {}) assert reason is not None assert resolved == () @@ -3143,7 +3277,7 @@ def test_git_restore_paths_denies_an_unrecognized_flag(flag: str) -> None: assume(flag not in checker._RESTORE_BOOLEAN_FLAGS | checker._RESTORE_VALUE_FLAGS) assume(not flag.startswith("--pathspec-from-file") and not flag.startswith("--recurse-submodules")) assume(flag not in ("--staged", "--no-staged", "--worktree", "--no-worktree")) - reason, resolved = checker._git_restore_paths([flag], {}) + reason, resolved = checker._git_restore_paths([flag], {}, {}) assert reason is not None assert resolved == () @@ -3617,7 +3751,7 @@ def test_git_checkout_paths_folds_branch_creation_flags_into_the_non_goal(flag: discards the change while the old code reported this as checked-safe. Must now fold into the same honest, no-claim Non-goal bare `git checkout SOMENAME` already carries -- empty paths, not a false claim.""" - reason, paths = checker._git_checkout_paths([flag, "newbranch", "other"], {}) + reason, paths = checker._git_checkout_paths([flag, "newbranch", "other"], {}, {}) assert reason is None assert paths == () @@ -3627,7 +3761,7 @@ def test_git_checkout_paths_branch_creation_flag_wins_even_with_a_double_dash() exclusive with every pathspec-checkout mode (per `git checkout -h`'s own synopsis) -- the Non-goal fold must fire before sub-case (a)'s own `--`-present branch is ever reached, not only when `--` is absent.""" - reason, paths = checker._git_checkout_paths(["-b", "newbranch", "--", "file.py"], {}) + reason, paths = checker._git_checkout_paths(["-b", "newbranch", "--", "file.py"], {}, {}) assert reason is None assert paths == () @@ -3636,7 +3770,7 @@ def test_git_checkout_paths_still_extracts_a_real_path_without_a_branch_creation """No regression from the branch-creation fold: an ordinary two- positional pathspec checkout with no `-b`/`-B`/`--orphan` present is unaffected.""" - reason, paths = checker._git_checkout_paths(["a.py", "b.py"], {}) + reason, paths = checker._git_checkout_paths(["a.py", "b.py"], {}, {}) assert reason is None assert paths == ("a.py", "b.py") @@ -3663,7 +3797,7 @@ def test_git_checkout_paths_denies_pathspec_from_file(flag: str) -> None: a file containing the real pathspecs, not an ambiguous ref/path. Live-verified end-to-end that this silently discarded a dirty tracked file listed in the control file.""" - reason, resolved = checker._git_checkout_paths([flag, "files.txt"], {}) + reason, resolved = checker._git_checkout_paths([flag, "files.txt"], {}, {}) assert reason is not None assert resolved == () @@ -3683,7 +3817,7 @@ def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_pat command (`git checkout -- a.py; git restore b.py`) accumulate paths from every segment, not just the first.""" segments = [["git", "checkout", "--", *command_paths], ["git", "restore", *command_paths]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) assert reason is None assert resolved == (*command_paths, *command_paths) @@ -3696,7 +3830,7 @@ def test_rule_git_checkout_restore_denies_when_git_dir_env_var_assigned() -> Non token-shape fact) rather than letting the live wrapper check the wrong tree.""" segments = [["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}, {}, {}, {}) assert reason is not None assert resolved == () @@ -3707,7 +3841,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_cd() -> Non the wrapper's own fixed `.cwd` unsound for a LATER checkout/restore segment -- denied outright.""" segments = [["cd", "/tmp"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) assert reason is not None assert resolved == () @@ -3717,7 +3851,7 @@ def test_rule_git_checkout_restore_allows_cd_after_the_checkout_segment() -> Non `cd` in an EARLIER segment -- a `cd` AFTER the checkout/restore segment does not retroactively make the already-scanned segment unsound.""" segments = [["git", "checkout", "--", "f.py"], ["cd", "/tmp"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) assert reason is None assert resolved == ("f.py",) @@ -3733,7 +3867,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_pushd_or_po claim that the wrapper's live check then found clean at the wrong `.cwd`, silently allowing a real, uncommitted-change discard.""" segments = [[relocator, "/tmp"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) assert reason is not None assert resolved == () @@ -3758,7 +3892,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_starts_with_a_ `checkout_restore_paths` claim the same way round 9's own fix closed for the literal case.""" segments = [["$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}) assert reason is not None assert resolved == () @@ -3770,7 +3904,7 @@ def test_rule_git_checkout_restore_allows_a_genuinely_vanishing_dynamic_word() - real bash would run whatever token follows as the actual command word instead, and that token is scanned on its own merits.""" segments = [["${NEVERSET}", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) assert reason is None assert resolved == ("f.py",) @@ -3814,7 +3948,7 @@ def test_rule_git_checkout_restore_allows_a_dynamic_word_resolving_to_something_ not a live production gap, since production always keeps the two dicts' own key sets in sync).""" segments = [["$EDITOR", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}, {}, {"EDITOR": "vim"}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}, {}, {"EDITOR": "vim"}, {}) assert reason is None assert resolved == ("f.py",) @@ -3893,7 +4027,7 @@ def test_dynamic_word_may_resolve_to_a_cwd_relocator_true_for_a_still_dynamic_ca def test_rule_git_checkout_restore_denies_a_still_dynamic_candidate() -> None: segments = [["${UNSET:-$OTHER}", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}, {}, {}, {}) assert reason is not None assert resolved == () @@ -3928,7 +4062,7 @@ def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_v assigned) resolved to a CONFIDENT, WRONG `checkout_restore_paths` claim -- real bash genuinely runs `cd sub` there.""" segments = [["$NEVERSET", "$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}) assert reason is not None assert resolved == () @@ -3971,7 +4105,7 @@ def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_r Live-verified before this fix: `X=cd; > /dev/null $X sub; git checkout -- dirty.py` resolved to a confident, wrong ALLOW.""" segments = [[">", "/dev/null", "$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}) assert reason is not None assert resolved == () @@ -4105,13 +4239,13 @@ def test_git_checkout_paths_excludes_a_trailing_redirect_clause() -> None: wrapper check to wrongly deny whenever the unrelated append target happened to be dirty, even though an append redirect can never discard that file's existing content.""" - deny_reason, paths = checker._git_checkout_paths(["--", "f.py", ">>", "unrelated_append_target.py"], {}) + deny_reason, paths = checker._git_checkout_paths(["--", "f.py", ">>", "unrelated_append_target.py"], {}, {}) assert deny_reason is None assert paths == ("f.py",) def test_git_restore_paths_excludes_a_trailing_redirect_clause() -> None: - deny_reason, paths = checker._git_restore_paths(["f.py", ">>", "unrelated_append_target.py"], {}) + deny_reason, paths = checker._git_restore_paths(["f.py", ">>", "unrelated_append_target.py"], {}, {}) assert deny_reason is None assert paths == ("f.py",) @@ -4149,7 +4283,7 @@ def test_git_checkout_paths_does_not_drop_a_digit_shaped_path() -> None: real, dirty, tracked file) -- the classifier's own former digit- consuming redirect heuristic wrongly treated `2` as an fd-redirect prefix rather than a real path argument.""" - deny_reason, paths = checker._git_checkout_paths(["--", "realfile.py", "2", ">", "target.txt"], {}) + deny_reason, paths = checker._git_checkout_paths(["--", "realfile.py", "2", ">", "target.txt"], {}, {}) assert deny_reason is None assert paths == ("realfile.py", "2") @@ -4161,7 +4295,7 @@ def test_git_restore_paths_does_not_drop_a_real_path_behind_a_digit_redirect() - once `2` vanished into the wrongly-recognized redirect, `--source`'s own value-consumption swallowed `file.py` itself, the actual restore target, leaving nothing for the live wrapper check to examine.""" - deny_reason, paths = checker._git_restore_paths(["--source", "2", ">", "target.txt", "file.py"], {}) + deny_reason, paths = checker._git_restore_paths(["--source", "2", ">", "target.txt", "file.py"], {}, {}) assert deny_reason is None assert paths == ("file.py",) From d14f75ca3708911fc0efbc64fb14ba8d21bac6a4 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 19:18:59 +0000 Subject: [PATCH 27/46] fix(hooks): close the reassignment-ambiguity bypass on two HARD-DENY paths A fresh, independent adversarial review of this PR's current head (round 22) found the same order-blind reassignment-ambiguity class rounds 19-21 already closed for the checkout/restore consumer (git-token recognition, cd/pushd/popd relocation, and dynamic path- argument resolution) was still open on two entirely different, HARD- DENY consumers: B1a/B1b's own tool+verb dynamic-indirection detection (guarding `pip`/`uv install`-shaped invocations) and `_rule_gh_api_write`'s own dynamic -X/--method and -f/-F/--field/ --raw-field resolution (guarding `gh api` write calls, including a PR merge). Both were fed the ordinary, order-blind `assigned`/ `raw_assigned` dicts, with no bias mechanism at all. Independently re-verified live against the current source before acting: `A=uv; B=install; $A $B foo; B=somethingelse` and `M=POST; gh api repos/o/r/pulls/1/merge -X $M; M=safe` both classified `deny=False`, while a real bash proxy (stand-in `uv`/`gh` binaries on PATH, capturing their own argv) confirmed real bash genuinely ran `uv install foo` and `gh api .../merge -X POST` respectively -- a genuine, unrecognized package install and a genuine, unrecognized write API call, not merely a missed advisory warning. This directly contradicts `_assigned_raw_values_biased_toward`'s own prior docstring claim that every consumer other than checkout/restore only risked "a missed advisory warning or an unrecognized non-destructive write, not irreversible data loss" -- corrected in place alongside this fix. Closed the same way rounds 19/20 closed their own consumers, mirroring round 20's own `_CWD_RELOCATING_COMMANDS` approach: a new `_WATCHED_WRITE_BIAS` combines every literal both consumers care about (`_WATCHED_TOOLS`, `_WATCHED_VERBS`, the git-push verb, `_WRITE_ METHODS`, and the gh-api flag-name literals) into one shared bias target set -- safe because the bias mechanism only decides whether a NAME's own resolution STICKS to a qualifying value; every actual comparison downstream still filters against its own narrower target set unchanged. A new `_assigned_literals_biased_toward` is the lowercased counterpart of the existing `_assigned_raw_values_biased_ toward`, needed because `_rule_gh_api_write`/B1a/B1b resolve through `_substitute_var_refs_candidates`, which takes both a lowercased name_to_value and a raw-case name_to_raw_value. Every call site that resolves a dynamic token for `_rule_gh_api_write` or `_segment_loop_hit` (B1a/B1b) now tries the ordinary reading first, OR-ed with a second attempt against the write-biased dicts -- for both the top-level segment scope AND the leading-decoy-collapsed segment scope `_segment_loop_hit`'s own second pass already runs. Learning from round 20's own initial under-scoping (later corrected in round 21 after an adversarial review found the gap live), the two new outer-scope parameters are threaded through the FULL recursive chain from this round onward, not scoped down to the top-level invocation only: `classify()`, `_classify_tokens`, `_rule_command_substitution_ content`, and `_rule_array_literal_content` all gained the two new parameters. Independently confirmed live that this closes the substitution-boundary-straddling shape too: `A=uv; x=$($A install foo); A=somethingelse` and `M=POST; x=$(gh api repos/o/r/pulls/1/merge -X $M); M=safe` -- the tool+verb/method pair used entirely WITHIN the `$(...)` span, with the reassignment-ambiguity poisoning the OUTER token stream -- both now correctly deny. Verified the detection-logic property-coverage gate reports no new trigger requiring further coverage (the new helper delegates entirely to the already-tested `_assigned_raw_values_biased_toward`, and the call-site changes are pure OR-composition of already-tested rule functions, adding no new regex/comparison logic of their own) -- run against the uncommitted working tree (`git diff -- '*.py'`, no HEAD ref), the CORRECT methodology this session's own round 21 established. Regression tests added at every established layer regardless: end-to-end wrapper-level (hooks/test_gitapex_check_ bash_safety.py, both reproductions plus the command-substitution variant), unit/property level for the new helper, and classify()-level for both the top-level and command-substitution shapes plus a false- positive guard (tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI thresholds: --max-absolute E --max-modules B --max-average A), the property- coverage gate, and no coverage regression on hooks/gitapex_check_ bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-22 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 210 ++++++++++++++++-- hooks/test_gitapex_check_bash_safety.py | 28 +++ ...st_gitapex_check_bash_safety_properties.py | 137 ++++++++++-- 3 files changed, 344 insertions(+), 31 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index fc2f2fb1..24a49cd6 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -853,6 +853,8 @@ def _rule_command_substitution_content( name_to_raw_value_git_biased: dict[str, str], name_to_raw_value_cd_biased: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]], + name_to_value_write_biased: dict[str, str], + name_to_raw_value_write_biased: dict[str, str], ) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `$(...)` command-substitution span's OWN inner content through this module's full rule set -- bash genuinely @@ -967,7 +969,24 @@ def _rule_command_substitution_content( issue #1375) are threaded the same way, for the analogous cd/pushd/ popd-relocation and path-argument reassignment bypasses -- see `_assigned_raw_values_biased_toward`'s own second CRITICAL-bug - paragraph and `_assigned_raw_value_history`'s own docstring.""" + paragraph and `_assigned_raw_value_history`'s own docstring. + + NAME_TO_VALUE_WRITE_BIASED and NAME_TO_RAW_VALUE_WRITE_BIASED (round + 22, issue #1375) are threaded the same way, for the analogous B1a/B1b + tool+verb and `_rule_gh_api_write` method/field reassignment bypasses + on THIS module's own HARD-DENY paths -- see `_assigned_literals_ + biased_toward`'s own docstring for the live bypass this closes and + `_classify_tokens`'s own docstring for where the OR-fallback calls + that actually consume these two dicts live (`_rule_gh_api_write` and + `_segment_loop_hit` are both called against SEGMENTS derived from the + top-level token stream, not against this function's own recursive + inner content directly -- these two parameters exist here only so a + `pip install`/`gh api` write hidden ENTIRELY WITHIN a `$(...)` span, + with the reassignment-ambiguity poisoning the OUTER token stream the + same way round 21 found for the cd-biased case, still resolves + correctly once the inner content reaches ITS OWN recursive + `_classify_tokens`/`classify` call's own local `_rule_gh_api_write`/ + `_segment_loop_hit` invocations).""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -1002,6 +1021,8 @@ def _rule_command_substitution_content( name_to_raw_value_git_biased, name_to_raw_value_cd_biased, name_to_raw_value_history, + name_to_value_write_biased, + name_to_raw_value_write_biased, ) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) @@ -1023,6 +1044,8 @@ def _rule_command_substitution_content( name_to_raw_value_git_biased, name_to_raw_value_cd_biased, name_to_raw_value_history, + name_to_value_write_biased, + name_to_raw_value_write_biased, ) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) @@ -1680,13 +1703,26 @@ def _assigned_raw_values_biased_toward(tokens: list[str], literals: frozenset[st -- never toward silently missing a real git invocation, which is the unsafe direction here). Used to feed the outer git-token-recognition fallback in `_find_git_checkout_restore`, AND (round 20, below) the - cd/pushd/popd-relocation fallback in `_rule_git_checkout_restore` -- - every OTHER consumer of `name_to_raw_value` in this module keeps + cd/pushd/popd-relocation fallback in `_rule_git_checkout_restore`. + + CORRECTION (round 22, issue #1375): this paragraph previously claimed + every OTHER consumer of `name_to_raw_value` in this module could keep using the ordinary, order-blind `_assigned_raw_values` unchanged, - since a reassignment-ambiguity miss for one of those risks a missed - advisory warning or an unrecognized non-destructive write, not + since a reassignment-ambiguity miss for one of those risked only a + missed advisory warning or an unrecognized non-destructive write, not irreversible data loss -- the same reasoning that scoped round 19's - own original, narrower fix. + own original, narrower fix. That claim was FALSE for two consumers: + B1a/B1b's own tool+verb reconstruction (guarding `pip`/`uv + install`-shaped invocations) and `_rule_gh_api_write`'s own dynamic + -X/--method and -f/-F/--field/--raw-field resolution (guarding `gh + api` write calls, including a PR merge) are both HARD-DENY paths, not + advisory-only, and a reassignment-ambiguity miss on either is a + genuine, unrecognized package install or unreviewed write API call -- + see `_assigned_literals_biased_toward`'s own docstring for the live + reproduction and fix. The claim remains accurate for this module's + remaining `name_to_raw_value` consumers not covered by that fix (e.g. + the warn-only `git push` detection, and the graphql-mutation-keyword + substring residual disclosed in this module's own header docstring). CRITICAL bug found by independent adversarial review (round 20, issue #1375) and independently reproduced live: `_rule_git_checkout_ @@ -1755,6 +1791,65 @@ def _assigned_raw_values_biased_toward(tokens: list[str], literals: frozenset[st return values +def _assigned_literals_biased_toward(tokens: list[str], literals: frozenset[str]) -> dict[str, str]: + """Lowercased counterpart of `_assigned_raw_values_biased_toward` + above, for a consumer that needs the `name_to_value`-shaped + (already-lowercased) dict as its own resolution source -- e.g. + `_substitute_var_refs_candidates`'s own first argument -- not the + raw-case dict that function itself returns. A thin lowering wrapper + around the same bias computation, not a separate one of its own, so + both dicts stay in lockstep for the same NAME by construction; see + `_assigned_raw_values_biased_toward`'s own docstring for the bias + rule this applies (sticks to the first value seen that is a member of + LITERALS, case-insensitively, ignoring any later, different + reassignment). + + CRITICAL bug found by independent adversarial review (round 22, issue + #1375) and independently reproduced live: `_rule_b1a_dynamic_word_ + same_segment_verb`/`_rule_b1b_dynamic_word_assigned_tool_and_verb` + (B1a/B1b, the tool+verb dynamic-indirection HARD-DENY rules guarding + `pip`/`uv install`-shaped invocations) and every `_rule_gh_api_write` + helper that resolves a dynamic `-X`/`--method` or `-f`/`--field` + value or flag name were, before this fix, all fed the ordinary, + order-blind `assigned`/`raw_assigned` dicts -- the SAME reassignment- + ambiguity class round 19 (git-token) and round 20 (cd/pushd/popd- + relocation) already closed for the checkout/restore consumer, left + open on these two entirely different, HARD-DENY consumers. `A=uv; + B=install; $A $B foo; B=somethingelse` resolved `$B` to `"somethingelse"` + (the LAST assignment in token order) even though `$B` genuinely was + `"install"` at its actual point of use one statement earlier; + `M=POST; gh api repos/o/r/pulls/1/merge -X $M; M=safe` resolved `$M` + the same way. Confirmed live end-to-end via a real bash proxy (stand-in + `uv`/`gh` binaries on PATH, capturing their own argv): both commands + were wrongly classified `deny=False` by this module, while real bash + genuinely executed `uv install foo` and `gh api .../merge -X POST` + respectively -- a genuine, unrecognized package install and a genuine, + unrecognized write API call (e.g. merging a pull request), not merely + a missed advisory warning. This directly contradicts `_assigned_raw_ + values_biased_toward`'s own prior claim (see that function's own + docstring, corrected alongside this fix) that every consumer other + than checkout/restore only risked "a missed advisory warning or an + unrecognized non-destructive write, not irreversible data loss" -- a + supply-chain package install and an unreviewed write to a live GitHub + repository (including a PR merge) are exactly the kind of consequence + that claim was meant to exclude. + + Closed the same way rounds 19/20 closed their own consumers: every + call site that resolves a dynamic token for B1a/B1b or `_rule_gh_api_ + write` now tries the ordinary reading first, OR-ed with a second + attempt against this bias mechanism fed `_WATCHED_WRITE_BIAS` (see + that constant's own comment for why one combined set covers both + consumers safely) -- see `_classify_tokens`'s own docstring for + exactly where the OR-fallback calls live and how the corresponding + raw-case bias dict (`_assigned_raw_values_biased_toward(tokens, + _WATCHED_WRITE_BIAS)`) is threaded alongside this one. Deliberately + NOT full execution-order tracking, the same bounded, one-directional, + safe-to-over-recognize posture every other bias fix in this module + already takes -- see `_assigned_raw_values_biased_toward`'s own + docstring for why that scoping is deliberate, not an oversight.""" + return {name: value.lower() for name, value in _assigned_raw_values_biased_toward(tokens, literals).items()} + + # Matches a token that is EXACTLY one bare `$NAME` or braced `${NAME}` # reference and nothing else -- no surrounding literal text, no default # clause, no indirect `${!NAME}` reference. Used only by `_resolve_path_ @@ -2537,6 +2632,8 @@ def _rule_array_literal_content( name_to_raw_value_git_biased: dict[str, str], name_to_raw_value_cd_biased: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]], + name_to_value_write_biased: dict[str, str], + name_to_raw_value_write_biased: dict[str, str], ) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `NAME=(...)` array-literal span's OWN inner content through this module's full rule set -- bash genuinely @@ -2649,7 +2746,13 @@ def _rule_array_literal_content( bypass it closes. NAME_TO_RAW_VALUE_CD_BIASED and NAME_TO_RAW_VALUE_ HISTORY (round 21, issue #1375) are threaded the same way -- see `_assigned_raw_values_biased_toward`'s own second CRITICAL-bug - paragraph and `_assigned_raw_value_history`'s own docstring.""" + paragraph and `_assigned_raw_value_history`'s own docstring. + NAME_TO_VALUE_WRITE_BIASED and NAME_TO_RAW_VALUE_WRITE_BIASED (round + 22, issue #1375) are threaded the same way -- see `_rule_command_ + substitution_content`'s own identical parameters' docstring paragraph + for what they mean and why they are threaded here even though this + function's own recursive call site does not itself invoke + `_rule_gh_api_write`/`_segment_loop_hit` directly.""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -2673,6 +2776,8 @@ def _rule_array_literal_content( name_to_raw_value_git_biased, name_to_raw_value_cd_biased, name_to_raw_value_history, + name_to_value_write_biased, + name_to_raw_value_write_biased, ) is_git_push = is_git_push or reading_verdict.is_git_push checkout_restore_paths.extend(reading_verdict.checkout_restore_paths) @@ -2793,6 +2898,29 @@ def _rule_array_literal_content( _WRITE_METHODS = {"post", "put", "patch", "delete"} +# The combined literal set B1a/B1b's own tool+verb reconstruction +# (`_rule_b1a_dynamic_word_same_segment_verb`/`_rule_b1b_dynamic_word_ +# assigned_tool_and_verb`) and `_rule_gh_api_write`'s own -X/--method and +# -f/-F/--field/--raw-field flag/value reconstruction each bias toward, +# for the same reassignment-ambiguity reason `_assigned_raw_values_biased_ +# toward`'s own docstring documents for the git-token and cd/pushd/popd- +# relocation consumers -- see `_assigned_literals_biased_toward`'s own +# docstring for the live bypass this closes on these two HARD-DENY paths. +# One shared, combined set (rather than a separate bias set per consumer) +# is safe: the bias mechanism only decides whether a NAME's own +# resolution STICKS to a qualifying value once assigned one; every actual +# comparison downstream still filters against its own narrower target set +# (`_WATCHED_TOOLS`, a specific VERB_SET, `_WRITE_METHODS`, or a specific +# flag-name literal) unchanged, so biasing toward the union cannot make a +# check fire on the wrong literal. +_WATCHED_WRITE_BIAS = frozenset( + _WATCHED_TOOLS + | _WATCHED_VERBS + | {_GIT_PUSH_VERB} + | _WRITE_METHODS + | {"-x", "--method", "-f", "--field", "--raw-field"} +) + class Verdict(NamedTuple): deny: bool @@ -4737,6 +4865,8 @@ def classify( outer_name_to_raw_value_git_biased: dict[str, str] | None = None, outer_name_to_raw_value_cd_biased: dict[str, str] | None = None, outer_name_to_raw_value_history: dict[str, tuple[str, ...]] | None = None, + outer_name_to_value_write_biased: dict[str, str] | None = None, + outer_name_to_raw_value_write_biased: dict[str, str] | None = None, ) -> Verdict: """Classify one Bash tool_input.command string. Fails closed (deny) on anything shlex cannot tokenize -- an unparseable command is exactly the @@ -4755,10 +4885,12 @@ def classify( OUTER_NAME_TO_RAW_VALUE_GIT_BIASED (round 19, issue #1375), OUTER_NAME_TO_RAW_VALUE_CD_BIASED and OUTER_NAME_TO_RAW_VALUE_HISTORY - (round 21, issue #1375) are the same recursive call's own analogous - further arguments -- see `_classify_tokens`'s own docstring and - `_find_git_checkout_restore`'s/`_assigned_raw_value_history`'s own - docstrings for what each means and the live bypass each closes.""" + (round 21, issue #1375), and OUTER_NAME_TO_VALUE_WRITE_BIASED/ + OUTER_NAME_TO_RAW_VALUE_WRITE_BIASED (round 22, issue #1375) are the + same recursive call's own analogous further arguments -- see + `_classify_tokens`'s own docstring and `_find_git_checkout_restore`'s/ + `_assigned_raw_value_history`'s/`_assigned_literals_biased_toward`'s + own docstrings for what each means and the live bypass each closes.""" try: tokens = tokenize(command) except TokenizeError as error: @@ -4770,6 +4902,8 @@ def classify( outer_name_to_raw_value_git_biased, outer_name_to_raw_value_cd_biased, outer_name_to_raw_value_history, + outer_name_to_value_write_biased, + outer_name_to_raw_value_write_biased, ) @@ -4780,6 +4914,8 @@ def _classify_tokens( outer_name_to_raw_value_git_biased: dict[str, str] | None = None, outer_name_to_raw_value_cd_biased: dict[str, str] | None = None, outer_name_to_raw_value_history: dict[str, tuple[str, ...]] | None = None, + outer_name_to_value_write_biased: dict[str, str] | None = None, + outer_name_to_raw_value_write_biased: dict[str, str] | None = None, ) -> Verdict: """The token-level core of `classify` -- split out so `_rule_command_ substitution_content` can recurse into a `$(...)` span's own inner @@ -4836,12 +4972,28 @@ def _classify_tokens( resolution bypass round 20's own scoped-down (non-recursively- threaded) cd-biased fix did NOT cover -- see `_assigned_raw_values_ biased_toward`'s own second CRITICAL-bug paragraph and `_assigned_ - raw_value_history`'s own docstring for both bypasses.""" + raw_value_history`'s own docstring for both bypasses. + + OUTER_NAME_TO_VALUE_WRITE_BIASED and OUTER_NAME_TO_RAW_VALUE_ + WRITE_BIASED (round 22, issue #1375) are two further, parallel + outer-scope arguments, merged the identical way -- `_assigned_ + literals_biased_toward(tokens, _WATCHED_WRITE_BIAS)` and `_assigned_ + raw_values_biased_toward(tokens, _WATCHED_WRITE_BIAS)` respectively -- + and threaded the same way into the same two recursive calls. Unlike + every prior bias pair above, these two are NOT threaded into `_rule_ + git_checkout_restore` (that rule has no B1a/B1b/gh-api-write concern + at all); instead they feed a second, OR-ed attempt at `_rule_gh_api_ + write` and `_segment_loop_hit` below, alongside the ordinary ASSIGNED/ + RAW_ASSIGNED attempt -- see `_assigned_literals_biased_toward`'s own + docstring for the live bypass this closes on those two HARD-DENY + consumers specifically.""" outer_literals = outer_name_to_value or {} outer_raw = outer_name_to_raw_value or {} outer_raw_git_biased = outer_name_to_raw_value_git_biased or {} outer_raw_cd_biased = outer_name_to_raw_value_cd_biased or {} outer_raw_history = outer_name_to_raw_value_history or {} + outer_write_biased = outer_name_to_value_write_biased or {} + outer_raw_write_biased = outer_name_to_raw_value_write_biased or {} merged_name_to_value = {**outer_literals, **_assigned_literals(tokens)} merged_name_to_raw_value = {**outer_raw, **_assigned_raw_values(tokens)} merged_name_to_raw_value_git_biased = { @@ -4855,6 +5007,14 @@ def _classify_tokens( merged_name_to_raw_value_history = _merge_raw_value_histories( outer_raw_history, _assigned_raw_value_history(tokens) ) + merged_name_to_value_write_biased = { + **outer_write_biased, + **_assigned_literals_biased_toward(tokens, _WATCHED_WRITE_BIAS), + } + merged_name_to_raw_value_write_biased = { + **outer_raw_write_biased, + **_assigned_raw_values_biased_toward(tokens, _WATCHED_WRITE_BIAS), + } content_reason, content_is_git_push, content_checkout_restore_paths = _rule_command_substitution_content( tokens, @@ -4863,6 +5023,8 @@ def _classify_tokens( merged_name_to_raw_value_git_biased, merged_name_to_raw_value_cd_biased, merged_name_to_raw_value_history, + merged_name_to_value_write_biased, + merged_name_to_raw_value_write_biased, ) if content_reason: return Verdict(True, content_reason, content_is_git_push, content_checkout_restore_paths) @@ -4874,6 +5036,8 @@ def _classify_tokens( merged_name_to_raw_value_git_biased, merged_name_to_raw_value_cd_biased, merged_name_to_raw_value_history, + merged_name_to_value_write_biased, + merged_name_to_raw_value_write_biased, ) is_git_push = content_is_git_push or array_content_is_git_push checkout_restore_paths = content_checkout_restore_paths + array_content_checkout_restore_paths @@ -4893,6 +5057,14 @@ def _classify_tokens( **_assigned_raw_values_biased_toward(tokens, _CWD_RELOCATING_COMMANDS), } raw_assigned_history = _merge_raw_value_histories(outer_raw_history, _assigned_raw_value_history(tokens)) + assigned_write_biased = { + **outer_write_biased, + **_assigned_literals_biased_toward(tokens, _WATCHED_WRITE_BIAS), + } + raw_assigned_write_biased = { + **outer_raw_write_biased, + **_assigned_raw_values_biased_toward(tokens, _WATCHED_WRITE_BIAS), + } lowered_command = " ".join(tokens).lower() is_git_push = is_git_push or any(_is_git_push_segment(seg, raw_assigned) for seg in segments) @@ -4901,12 +5073,19 @@ def _classify_tokens( if literal_hit: return Verdict(True, literal_hit, is_git_push, checkout_restore_paths) - gh_api_hit = _rule_gh_api_write(segments, lowered_command, assigned, raw_assigned) + gh_api_hit = _rule_gh_api_write(segments, lowered_command, assigned, raw_assigned) or _rule_gh_api_write( + segments, lowered_command, assigned_write_biased, raw_assigned_write_biased + ) if gh_api_hit: return Verdict(True, gh_api_hit, is_git_push, checkout_restore_paths) loop_hit, loop_is_git_push = _segment_loop_hit(segments, assigned, raw_assigned) is_git_push = is_git_push or loop_is_git_push + if not loop_hit: + loop_hit, biased_loop_is_git_push = _segment_loop_hit( + segments, assigned_write_biased, raw_assigned_write_biased + ) + is_git_push = is_git_push or biased_loop_is_git_push if loop_hit: return Verdict(True, loop_hit, is_git_push, checkout_restore_paths) @@ -4916,6 +5095,11 @@ def _classify_tokens( if collapsed_segments != segments: collapsed_hit, collapsed_is_git_push = _segment_loop_hit(collapsed_segments, assigned, raw_assigned) is_git_push = is_git_push or collapsed_is_git_push + if not collapsed_hit: + collapsed_hit, collapsed_biased_is_git_push = _segment_loop_hit( + collapsed_segments, assigned_write_biased, raw_assigned_write_biased + ) + is_git_push = is_git_push or collapsed_biased_is_git_push if collapsed_hit: return Verdict( True, diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index b85d8f99..bc570be0 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -864,6 +864,34 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "IFS=post; DECOY=POST; gh api repos/foo/bar/merge -X ${DECOY} extra", "gh-api-method-value-past-case-folded-ifs-collision", ), + # Found live by Step 8 independent review, twenty-second round (issue + # #1375, confirmed against issue #1326): B1a/B1b's own tool+verb + # reconstruction and `_rule_gh_api_write`'s own dynamic -X/--method + # resolution were both fed the ordinary, order-blind ASSIGNED/ + # RAW_ASSIGNED dicts -- the SAME reassignment-ambiguity class round + # 19 (git-token) and round 20 (cd/pushd/popd-relocation) already + # closed for the checkout/restore consumer, left open on these two + # entirely different, HARD-DENY consumers. Confirmed live via a real + # bash proxy (stand-in `uv`/`gh` binaries on PATH, capturing their own + # argv): `$B` genuinely was "install" at its actual point of use one + # statement earlier, and real bash genuinely ran `uv install foo`. + ("A=uv; B=install; $A $B foo; B=somethingelse", "var-split-tool-and-verb-reassigned-after-use"), + # Same round, the gh-api-write counterpart: `$M` genuinely was "POST" + # at its actual point of use; real bash genuinely ran `gh api + # repos/o/r/pulls/1/merge -X POST` -- a genuine, unreviewed write API + # call (e.g. merging a pull request). + ("M=POST; gh api repos/o/r/pulls/1/merge -X $M; M=safe", "gh-api-method-value-reassigned-after-use"), + # Round 22's own OR-fallback fix threads the write-biased dict through + # the SAME recursive chain rounds 19-21 already use, not merely the + # top-level segment scope -- confirms the reassignment straddling a + # command substitution's OWN boundary (the ambiguity living in the + # OUTER token stream, the tool+verb pair used entirely WITHIN the + # substitution) is closed too, mirroring round 21's own correction of + # round 20's initially-scoped-down cd-biased fix. + ( + "A=uv; x=$($A install foo); A=somethingelse", + "var-split-tool-and-verb-reassigned-after-use-across-command-substitution", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index fa1ca82b..72887cbd 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -1358,7 +1358,7 @@ def test_rule_command_substitution_content_detects_an_embedded_install(tool: str a punctuation character shlex breaks a word at, so an assignment's `NAME=` prefix stays fused onto the leading `$` in the same token.""" tokens = ["x=$", "(", tool, "install", "evil-pkg", ")"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None @@ -1374,7 +1374,7 @@ def test_rule_command_substitution_content_allows_harmless_inner_content(value: silently dropping a non-denying inner `is_git_push=True` signal (see the function's own docstring).""" tokens = ["echo", "$", "(", "date", value, ")"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) # --- Issue #1326 Stage 1, fifteenth round: bash's own leading-assignment ---- @@ -1493,7 +1493,7 @@ def test_rule_array_literal_content_detects_a_denied_pair_regardless_of_a_leadin `Y=1; A=(uv install $Y); "${A[@]}"` was wrongly ALLOWED before this function existed.""" tokens = ["dummy=", "(", f"${first}", "uv", "install", f"${second}", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None @@ -1512,7 +1512,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_bare_ref(unse fused with other text (not a bare whole-token reference), must NOT be collapsed -- that shape does not word-split away to nothing.""" tokens = ["dummy=", "(", f"${unset_name}", verb_a, "install", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None @@ -1532,13 +1532,13 @@ def test_rule_array_literal_content_allows_harmless_content() -> None: denied pattern, with or without a leading unassigned reference, stays allowed.""" tokens = ["dummy=", "(", "$NEVERSET", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_no_span_present() -> None: """Robustness: a token stream with no array-literal span at all (e.g. an ordinary command) returns cleanly, never a crash.""" - assert checker._rule_array_literal_content(["echo", "hi"], {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(["echo", "hi"], {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) def test_strip_leading_unassigned_bare_refs_stops_at_a_fused_token() -> None: @@ -1564,7 +1564,7 @@ def test_rule_array_literal_content_empty_array_is_harmless() -> None: """No false positive / no crash: an empty array literal `NAME=()` has no inner content to recursively classify at all.""" tokens = ["dummy=", "(", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leading_unassigned_ref() -> None: @@ -1573,7 +1573,7 @@ def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leadin `_strip_leading_unassigned_bare_refs` to strip -- the collapsed reading equals the as-is one, so only one classification is needed.""" tokens = ["dummy=", "(", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> None: @@ -1590,7 +1590,7 @@ def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> No real, with a dynamic verb argument right after it -- exactly B2's own watched shape.""" tokens = ["dummy=", "(", "$NEVERSET", "uv", "$VERB", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None assert "unassigned reference" in reason @@ -1632,7 +1632,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_braced_bare_r this round, silently degrading the collapsed reading to a no-op for this shape.""" tokens = ["dummy=", "(", f"${{{unset_name}}}", verb_a, "install", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None @@ -1647,7 +1647,7 @@ def test_rule_array_literal_content_detects_an_outer_scope_resolved_pair() -> No recursive `_classify_tokens` call.""" tokens = ["dummy=", "(", "$G", "$P", "$M", ")"] outer = {"G": "gh", "P": "pr", "M": "merge"} - reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer, outer, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer, outer, {}, {}, {}, {}) assert reason is not None @@ -1683,7 +1683,9 @@ def test_rule_command_substitution_content_detects_an_outer_scope_resolved_check for the array-literal span.""" tokens = ["x=$", "(", "$G", "checkout", "--", "dirty.py", ")"] outer = {"G": "git"} - reason, _, checkout_restore_paths = checker._rule_command_substitution_content(tokens, outer, outer, outer, {}, {}) + reason, _, checkout_restore_paths = checker._rule_command_substitution_content( + tokens, outer, outer, outer, {}, {}, {}, {} + ) assert reason is None assert checkout_restore_paths == ("dirty.py",) @@ -2090,7 +2092,7 @@ def test_rule_array_literal_content_detects_a_braced_subscript_decoy() -> None: the subscript decoy blocked it from ever firing until it collapsed away.""" tokens = ["dummy=", "(", "${NEVERSET[0]}", "uv", "$VERB", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None @@ -2101,7 +2103,7 @@ def test_rule_array_literal_content_detects_a_fused_reference_chain_decoy() -> N before a fused chain of two bare references was recognized as vanishing as a unit.""" tokens = ["dummy=", "(", "$A_UNSET$B_UNSET", "gh", "pr", "merge", "1", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None @@ -2234,7 +2236,7 @@ def test_rule_command_substitution_content_scans_second_fused_span_in_same_token this test only proves that fix reached end-to-end through `_rule_command_substitution_content`'s own scan loop.""" tokens = ["echo", "$(echo ok)$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None @@ -2243,20 +2245,20 @@ def test_rule_command_substitution_content_skips_blank_fused_span_then_finds_den skipped without denying by itself, but scanning continues to the next fused span in the same token.""" tokens = ["echo", "$( )$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) assert reason is not None def test_rule_command_substitution_content_both_fused_spans_harmless() -> None: tokens = ["echo", "$(echo ok)$(echo also-ok)"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) def test_rule_command_substitution_content_empty_unquoted_span_skipped() -> None: """An empty, unquoted `$()` substitution has no inner tokens to recurse into -- distinct from the fused/quoted empty-span case above.""" tokens = ["$", "(", ")"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) def test_tokenize_raises_on_unbalanced_quote() -> None: @@ -4496,3 +4498,102 @@ def test_main_output_includes_empty_checkout_restore_paths_for_a_harmless_comman payload = {"tool_name": "Bash", "tool_input": {"command": "echo hi"}} out = _run_main(payload, monkeypatch, capsys) assert out["checkout_restore_paths"] == [] + + +def test_assigned_literals_biased_toward_stays_on_literal_once_assigned() -> None: + """Model-based, regression pin for the real bypass found live by Step + 8 independent review, twenty-second round (issue #1375): the + lowercased counterpart of `_assigned_raw_values_biased_toward` -- + once a name is assigned a value matching a member of LITERALS at any + point, it stays on that (lowercased) reading regardless of a later, + different reassignment, exactly mirroring the raw-case function's own + behavior.""" + assert checker._assigned_literals_biased_toward(["TOOL=UV", "TOOL=npm"], frozenset({"uv"})) == {"TOOL": "uv"} + + +def test_assigned_literals_biased_toward_falls_back_to_last_assignment_when_literal_never_seen() -> None: + """A name never assigned a biased-toward literal anywhere resolves + exactly as `_assigned_literals`'s own plain last-occurrence-wins + collapse would (lowercased).""" + assert checker._assigned_literals_biased_toward(["TOOL=NPM", "TOOL=Yarn"], frozenset({"uv"})) == {"TOOL": "yarn"} + + +def test_assigned_literals_biased_toward_matches_lowered_raw_bias() -> None: + """`_assigned_literals_biased_toward` is a thin lowering wrapper around + `_assigned_raw_values_biased_toward`, not a separate bias computation + of its own -- both dicts must agree for the same NAME, up to case, by + construction.""" + tokens = ["A=UV", "B=Install", "A=somethingelse"] + raw = checker._assigned_raw_values_biased_toward(tokens, checker._WATCHED_WRITE_BIAS) + literals = checker._assigned_literals_biased_toward(tokens, checker._WATCHED_WRITE_BIAS) + assert literals == {name: value.lower() for name, value in raw.items()} + + +@_PROPERTIES +@given(name=_IDENTIFIERS, decoy_value=_VALUES, tail=st.lists(_IDENTIFIERS, max_size=2)) +def test_assigned_literals_biased_toward_matches_plain_collapse_when_never_reassigned_to_literal( + name: str, decoy_value: str, tail: list[str] +) -> None: + """Model-based: for a single assignment never matching the biased- + toward literal, `_assigned_literals_biased_toward` agrees exactly with + the plain, order-blind `_assigned_literals`.""" + assume(decoy_value.lower() != "uv") + tokens = [f"{name}={decoy_value}", *tail] + assert checker._assigned_literals_biased_toward(tokens, frozenset({"uv"})) == checker._assigned_literals(tokens) + + +def test_classify_denies_tool_and_verb_indirection_when_verb_is_reassigned_after_use() -> None: + """End-to-end regression pin for the round-22 finding at the + `classify()` level, top-level shape: B1a/B1b's own tool+verb + reconstruction was fed the ordinary, order-blind ASSIGNED/RAW_ + ASSIGNED dicts, the SAME reassignment-ambiguity class rounds 19-20 + already closed for the checkout/restore consumer, left open on this + HARD-DENY consumer. Confirmed live via a real bash proxy (stand-in + `uv` binary on PATH): `$B` genuinely was "install" at its actual + point of use one statement earlier, and real bash genuinely ran `uv + install foo` -- but this module wrongly classified `deny=False` + before this fix.""" + verdict = checker.classify("A=uv; B=install; $A $B foo; B=somethingelse") + assert verdict.deny is True + + +def test_classify_denies_gh_api_write_when_method_value_is_reassigned_after_use() -> None: + """Companion to the tool+verb pin above, for `_rule_gh_api_write`'s + own dynamic -X/--method value resolution. Confirmed live via a real + bash proxy (stand-in `gh` binary on PATH): `$M` genuinely was "POST" + at its actual point of use; real bash genuinely ran `gh api + repos/o/r/pulls/1/merge -X POST` -- a genuine, unreviewed write API + call (e.g. merging a pull request) -- but this module wrongly + classified `deny=False` before this fix.""" + verdict = checker.classify("M=POST; gh api repos/o/r/pulls/1/merge -X $M; M=safe") + assert verdict.deny is True + + +def test_classify_denies_tool_and_verb_indirection_reassigned_across_a_command_substitution() -> None: + """End-to-end regression pin for round 22's own recursive-threading + correction, mirroring round 21's correction of round 20's initially + scoped-down cd-biased fix: the tool+verb pair is used entirely WITHIN + a `$(...)` span, but the reassignment-ambiguity poisoning lives in + the OUTER token stream -- `A=uv; x=$($A install foo); A=somethingelse` + -- so the write-biased dict must be threaded through the SAME + recursive chain as every other bias dict, not merely computed at the + top-level segment scope.""" + verdict = checker.classify("A=uv; x=$($A install foo); A=somethingelse") + assert verdict.deny is True + + +def test_classify_denies_gh_api_write_reassigned_across_a_command_substitution() -> None: + """Companion to the command-substitution pin above, for the gh-api- + write consumer.""" + verdict = checker.classify("M=POST; x=$(gh api repos/o/r/pulls/1/merge -X $M); M=safe") + assert verdict.deny is True + + +def test_classify_leaves_reassigned_but_unrelated_dynamic_word_allowed() -> None: + """No false positive: a name reassigned across statements, none of + whose values are ever a watched tool/verb/write-method, must not be + denied merely because it participates in this bias mechanism -- the + bias only ever widens toward a member of `_WATCHED_WRITE_BIAS`, never + invents a match out of nothing.""" + verdict = checker.classify("TOOL=echo; $TOOL hello; TOOL=world") + assert verdict.deny is False From 556013cd923bf07bc6364a747bb94c6b14934d0d Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 19:43:33 +0000 Subject: [PATCH 28/46] fix(hooks): widen checkout/restore path history to fused references A fresh, independent adversarial review of this PR's current head (round 23) found round 21's own history-widening fix for `_resolve_path_tokens` was scoped too narrowly: it only widened a token that is EXACTLY one bare/braced whole-token reference (`_BARE_OR_BRACED_VAR_REF_RE`), leaving a token that FUSES a reference with other text -- including the ordinary `"$DIR/$FILE"` path-join idiom -- to fall through to the ordinary, un-widened, order-blind resolution. Independently re-verified live before acting: `DIR=sub; FILE=dirty.py; git checkout -- "$DIR/$FILE"; DIR=other` resolved `checkout_restore_ paths` to `('other/dirty.py',)` alone -- a CONFIDENT, WRONG claim, not merely a missed one, since `$DIR` genuinely was `sub` at its actual point of use one statement earlier. Confirmed via a real bash proxy (stand-in `git` binary on PATH, capturing its own argv) that real bash resolves this to `checkout -- sub/dirty.py`, and end-to-end through the real wrapper against a scratch repo with `sub/dirty.py` genuinely dirty: the control command (no trailing reassignment) correctly denies citing `sub/dirty.py`; the same command with the trailing `DIR=other` wrongly allowed with exit 0, since the live `git diff --quiet` check ran against the wrong, nonexistent `other/dirty.py` and found it "clean" while the real, dirty file was never checked at all. Closed by generalizing the widening from "a token that IS one bare/braced reference" to "every name REFERENCED anywhere in a dynamic token that has more than one historical value": a new `_multi_valued_names_referenced` finds those names (covering braced, default-clause, indirect, and unbraced-with-prefix-ambiguity reference shapes, mirroring `_substitute_var_refs_candidates`'s own shape handling), and a new `_bounded_history_combinations` builds the cartesian product of one historical value per such name, bounded by the same `_MAX_SUBSTITUTION_CANDIDATES` this module's own quote-boundary expansion already uses (denying outright, not silently under-enumerating, when the product would exceed it). The existing, unchanged `_substitute_var_refs_candidates` is called once per combination -- still the sole authority on quote/fusion-aware resolution -- with every combination's own candidates unioned together. A name with only one historical value (or none) contributes no extra combinations, so a token referencing no multiply-assigned name resolves in exactly one pass, identical to before this fix -- no behavior change for the common case. The now-superseded `_BARE_OR_BRACED_VAR_REF_RE` special case is removed (the new mechanism is a strict superset: a bare/braced whole-token reference is just the degenerate single-name case), and `_assigned_raw_value_history`'s own docstring is corrected in place -- it previously described the fused-reference exclusion as a deliberate, safe narrowing; round 23 found it was itself the same order-blind- collapse bug class, just left open for that shape. Also corrects `.gitapex/ssot.json`'s own `bash-cli-write-and-install- guard` rule text (round 23's secondary finding): it named only "an earlier literal cd" as a cwd-relocation deny trigger, understating that the classifier also covers pushd/popd and a dynamic word resolvable to one of those via the existing reassignment-biased reading. Regression tests added at every established layer: end-to-end wrapper-level for both checkout and restore against a real scratch repo (hooks/test_gitapex_check_bash_safety.py), and unit-level for `_resolve_path_tokens`'s fused-reference widening, the two-name cartesian-product case, the single-history-value no-op case, and `_bounded_history_combinations`'s own empty-set and too-large-product cases, including the too-large-product path through `_resolve_path_tokens` itself for full coverage (tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI thresholds), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), the ssot.json drift scanner, and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-23 baseline). Refs #1375. --- .gitapex/ssot.json | 2 +- hooks/gitapex_check_bash_safety.py | 176 ++++++++++++++---- hooks/test_gitapex_check_bash_safety.py | 38 ++++ ...st_gitapex_check_bash_safety_properties.py | 79 +++++++- 4 files changed, 246 insertions(+), 49 deletions(-) diff --git a/.gitapex/ssot.json b/.gitapex/ssot.json index e33b8550..6b9ef00b 100644 --- a/.gitapex/ssot.json +++ b/.gitapex/ssot.json @@ -84,7 +84,7 @@ "id": "bash-cli-write-and-install-guard", "kind": "script", "script": "hooks/check-bash-safety.sh", - "rule": "Delegates classification to hooks/gitapex_check_bash_safety.py, a token-based classifier (Python stdlib shlex, POSIX mode) that matches against bash's own dequoted, operator-segmented token stream -- closing the quote-splitting, ${IFS}/$IFS substitution, and variable/array/positional-parameter indirection bypass classes a prior raw-text regex substring scan was live-confirmed vulnerable to (issue #1326). Denies a Bash call matching a package/plugin-install verb (uv add/uv remove and apm install/apm uninstall stay allowed as declarative, visibly-mutating commands); denies gh issue/gh pr write subcommands and gh api writes (POST/PUT/PATCH/DELETE, a field flag, or 'mutation' in gh api graphql); on git push (including an obfuscated/indirected one that still resolves to git push), runs skills/outward-artifact-preflight/scripts/gitapex_scan_provenance.py against the outgoing commit range and warns (never blocks) if it flags something. Since issue #1375: the classifier also extracts every path a git checkout/git restore invocation could discard (git checkout -- PATH, git checkout ./.., git checkout with 2+ positionals and no --, or git restore PATH/--staged --worktree PATH, a case-sensitive enumerated restore flag vocabulary), denying outright (no live I/O) when a path token is dynamic and cannot be soundly resolved to a literal, when an otherwise-risky shape yields zero paths (e.g. bare 'git checkout --'), when a restore segment carries --pathspec-from-file/--pathspec-file-nul or an unrecognized flag, or when the classifier cannot soundly determine which working tree is at risk (a -C/--git-dir/--work-tree flag, a GIT_DIR=/GIT_WORK_TREE=/GIT_INDEX_FILE= assignment, or an earlier literal cd in the same command); when paths are extracted, hooks/check-bash-safety.sh reads .cwd from the PreToolUse payload itself (not $CLAUDE_PROJECT_DIR) and denies if `git -C \"$cwd\" diff --quiet -- \"$path\"` reports any of them dirty. Fails closed (denies) rather than allowing the call through when jq or python3 is missing from PATH, the payload or tool_input is not a JSON object, tool_name is present but not a string, tool_input.command is present but not a string, the classifier exits non-zero, the classifier's own output is not a JSON object with a recognized decision, .cwd is missing/not a git working tree, or a checkout/restore path's own live git-diff check cannot be verified. Disclosed residual (not closed by this classifier): verb-token-splitting that never places the tool/verb name as its own literal token anywhere, e.g. string-slice reconstruction (cmd=uvinstall; eval \"${cmd:0:2} ${cmd:2}\") or array-literal-assignment indirection (A=(uv); V=(install); \"${A[@]}\" \"${V[@]}\") -- see hooks/gitapex_check_bash_safety.py's own module docstring. A bare 'git checkout SOMENAME' (single positional, not '.'/'..', no --) is a deliberate Non-goal: disambiguating a branch/ref name from a path needs a live ref-existence lookup this pure classifier does not perform.", + "rule": "Delegates classification to hooks/gitapex_check_bash_safety.py, a token-based classifier (Python stdlib shlex, POSIX mode) that matches against bash's own dequoted, operator-segmented token stream -- closing the quote-splitting, ${IFS}/$IFS substitution, and variable/array/positional-parameter indirection bypass classes a prior raw-text regex substring scan was live-confirmed vulnerable to (issue #1326). Denies a Bash call matching a package/plugin-install verb (uv add/uv remove and apm install/apm uninstall stay allowed as declarative, visibly-mutating commands); denies gh issue/gh pr write subcommands and gh api writes (POST/PUT/PATCH/DELETE, a field flag, or 'mutation' in gh api graphql); on git push (including an obfuscated/indirected one that still resolves to git push), runs skills/outward-artifact-preflight/scripts/gitapex_scan_provenance.py against the outgoing commit range and warns (never blocks) if it flags something. Since issue #1375: the classifier also extracts every path a git checkout/git restore invocation could discard (git checkout -- PATH, git checkout ./.., git checkout with 2+ positionals and no --, or git restore PATH/--staged --worktree PATH, a case-sensitive enumerated restore flag vocabulary), denying outright (no live I/O) when a path token is dynamic and cannot be soundly resolved to a literal, when an otherwise-risky shape yields zero paths (e.g. bare 'git checkout --'), when a restore segment carries --pathspec-from-file/--pathspec-file-nul or an unrecognized flag, or when the classifier cannot soundly determine which working tree is at risk (a -C/--git-dir/--work-tree flag, a GIT_DIR=/GIT_WORK_TREE=/GIT_INDEX_FILE= assignment, or an earlier cd/pushd/popd -- literal, or dynamic and resolvable to one via a reassignment-biased reading -- in the same command); when paths are extracted, hooks/check-bash-safety.sh reads .cwd from the PreToolUse payload itself (not $CLAUDE_PROJECT_DIR) and denies if `git -C \"$cwd\" diff --quiet -- \"$path\"` reports any of them dirty. Fails closed (denies) rather than allowing the call through when jq or python3 is missing from PATH, the payload or tool_input is not a JSON object, tool_name is present but not a string, tool_input.command is present but not a string, the classifier exits non-zero, the classifier's own output is not a JSON object with a recognized decision, .cwd is missing/not a git working tree, or a checkout/restore path's own live git-diff check cannot be verified. Disclosed residual (not closed by this classifier): verb-token-splitting that never places the tool/verb name as its own literal token anywhere, e.g. string-slice reconstruction (cmd=uvinstall; eval \"${cmd:0:2} ${cmd:2}\") or array-literal-assignment indirection (A=(uv); V=(install); \"${A[@]}\" \"${V[@]}\") -- see hooks/gitapex_check_bash_safety.py's own module docstring. A bare 'git checkout SOMENAME' (single positional, not '.'/'..', no --) is a deliberate Non-goal: disambiguating a branch/ref name from a path needs a live ref-existence lookup this pure classifier does not perform.", "planes": ["pretooluse"], "local_exclusion": "PreToolUse-only: grades a Claude Code tool-call JSON payload arriving on stdin, which has no working-tree equivalent to reconstruct ahead of a push.", "trigger": "PreToolUse matcher Bash (hooks/hooks.json)", diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 24a49cd6..f92e413e 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -482,6 +482,7 @@ class (rounds 5-8: shlex's own quote removal; round 9: bash's default- from __future__ import annotations +import itertools import json import re import shlex @@ -1850,17 +1851,6 @@ def _assigned_literals_biased_toward(tokens: list[str], literals: frozenset[str] return {name: value.lower() for name, value in _assigned_raw_values_biased_toward(tokens, literals).items()} -# Matches a token that is EXACTLY one bare `$NAME` or braced `${NAME}` -# reference and nothing else -- no surrounding literal text, no default -# clause, no indirect `${!NAME}` reference. Used only by `_resolve_path_ -# tokens`'s own history-widening (see `_assigned_raw_value_history`'s own -# docstring): for this narrow, unambiguous whole-token shape, the token's -# real bash value is exactly NAME's own raw value, with no risk of the -# unbraced-prefix ambiguity `_substitute_var_refs_candidates` itself -# exists to handle for a token containing MORE than just the reference. -_BARE_OR_BRACED_VAR_REF_RE = re.compile(r"^\$([A-Za-z_][A-Za-z0-9_]*)$|^\$\{([A-Za-z_][A-Za-z0-9_]*)\}$") - - def _assigned_raw_value_history(tokens: list[str]) -> dict[str, tuple[str, ...]]: """Like `_assigned_raw_values`, but maps each assigned name to the TUPLE of every DISTINCT raw value assigned to it anywhere in TOKENS @@ -1897,20 +1887,31 @@ def _assigned_raw_value_history(tokens: list[str]) -> dict[str, tuple[str, ...]] `cd`/`pushd`/`popd` -- an arbitrary path has no small, enumerable target set), so the fix this history dict feeds is different in kind from `_assigned_raw_values_biased_toward`'s own single-reading bias: - `_resolve_path_tokens` extracts EVERY distinct historical value for a - name referenced by a bare/braced whole-token reference (see - `_BARE_OR_BRACED_VAR_REF_RE`) as its own SEPARATE candidate path, - rather than picking one. This over-includes (an extra, harmless + `_resolve_path_tokens` extracts EVERY distinct historical value for + every name a dynamic token references as its own SEPARATE candidate + path, rather than picking one. This over-includes (an extra, harmless candidate path gets checked for dirtiness) rather than ever silently dropping the one that matters -- the same safe direction every other - ambiguity in this module resolves toward. Deliberately narrower than - full soundness: a token that FUSES a reference with literal text, a - default-value clause, or an indirect `${!NAME}` reference is NOT - widened by this mechanism and keeps whatever single reading - `_substitute_var_refs_candidates` already gives it -- the same - "handle the common, simple bare-reference shape narrowly, don't - attempt full generality" scoping this module's own reassignment-bias - fixes already use elsewhere.""" + ambiguity in this module resolves toward. + + CORRECTION (round 23, issue #1375): this paragraph previously claimed + the widening was deliberately narrowed to a token that is EXACTLY one + bare/braced whole-token reference, with a token that FUSES a + reference with literal text, a default-value clause, or an indirect + `${!NAME}` reference left un-widened as an accepted scoping choice. + That scoping was not merely narrower-but-safe -- it was itself the + identical order-blind-collapse bug class, just left open for the + fused case: `DIR=sub; FILE=dirty.py; git checkout -- "$DIR/$FILE"; + DIR=other` (the ordinary path-join idiom) produced a CONFIDENT, WRONG + `checkout_restore_paths` claim, live-confirmed to silently discard + real uncommitted work -- see `_resolve_path_tokens`'s own docstring + for the full reproduction and fix. Every name a dynamic token + references (bare, braced, default-clause, or indirect, including + every reference fused with other text in the same token) with more + than one historical value is now widened, via the cartesian product + of `_multi_valued_names_referenced`'s own found names + (`_bounded_history_combinations`, bounded the same way `_substitute_ + var_refs_candidates`'s own quote-boundary expansion already is).""" history: dict[str, list[str]] = {} for token in tokens: if _is_dynamic(token): @@ -3669,6 +3670,55 @@ def _is_git_push_segment(seg: list[str], name_to_raw_value: dict[str, str]) -> b _CHECKOUT_BRANCH_CREATION_FLAGS = {"-b", "-B", "--orphan"} +def _multi_valued_names_referenced(token: str, name_to_raw_value_history: dict[str, tuple[str, ...]]) -> set[str]: + """Every variable NAME referenced anywhere in TOKEN -- braced + (`${NAME}`), default-clause (`${NAME:-x}`), indirect (`${!NAME}`), or + unbraced (`$NAME`, including every non-empty PREFIX of an unbraced + run, mirroring `_unbraced_ref_options`'s own prefix enumeration, + since an unbraced reference is ambiguous about where the name ends) + -- whose own NAME_TO_RAW_VALUE_HISTORY entry carries MORE than one + distinct value. A name assigned only once (or never) is excluded: its + single reading already matches the ordinary, un-widened resolution, + so there is nothing to widen. Used by `_resolve_path_tokens`'s own + fused-reference history widening (see that function's own docstring) + to find which names' history actually needs enumerating, not to + resolve the token itself -- `_substitute_var_refs_candidates` still + does the real, quote/fusion-aware resolution, once per combination.""" + names: set[str] = set() + for match in _VAR_REF_FULL_RE.finditer(token): + braced_name, default_name, _default_text, indirect_name, unbraced_run = match.groups() + candidate_names = [n for n in (braced_name, default_name, indirect_name) if n is not None] + if unbraced_run is not None: + candidate_names.extend(unbraced_run[:i] for i in range(len(unbraced_run), 0, -1)) + for name in candidate_names: + if len(name_to_raw_value_history.get(name, ())) > 1: + names.add(name) + return names + + +def _bounded_history_combinations( + names: set[str], name_to_raw_value_history: dict[str, tuple[str, ...]] +) -> list[dict[str, str]] | None: + """Every combination of one historical value per NAME in NAMES, as a + NAME -> value override dict ready to merge over an existing + NAME_TO_RAW_VALUE for one `_substitute_var_refs_candidates` call per + combination -- the cartesian product across each name's own history + tuple. Returns `None` (too many combinations to enumerate soundly) if + the product would exceed `_MAX_SUBSTITUTION_CANDIDATES`, the same + bound and the same fail-closed convention `_substitute_var_refs_ + candidates`'s own quote-boundary expansion already uses -- an empty + NAMES returns `[{}]` (exactly one, no-op combination), never `None`, + so a caller with nothing to widen still gets a single pass through.""" + ordered = sorted(names) + histories = [name_to_raw_value_history[name] for name in ordered] + total = 1 + for history in histories: + total *= len(history) + if total > _MAX_SUBSTITUTION_CANDIDATES: + return None + return [dict(zip(ordered, combo, strict=True)) for combo in itertools.product(*histories)] + + def _resolve_path_tokens( tokens: list[str], name_to_raw_value: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]] ) -> tuple[str | None, tuple[str, ...]]: @@ -3704,21 +3754,70 @@ def _resolve_path_tokens( Confirmed live during this function's own development (before this check was added). - NAME_TO_RAW_VALUE_HISTORY (round 21, issue #1375) widens a BARE or - braced whole-token reference (`_BARE_OR_BRACED_VAR_REF_RE`) to every - DISTINCT value ever assigned to that name, not just the single, - possibly-stale one NAME_TO_RAW_VALUE's own order-blind collapse - gives -- see `_assigned_raw_value_history`'s own docstring for the - live bypass this closes. A token that is not exactly a bare/braced - reference (fused with literal text, a default clause, or an indirect - reference) keeps the ordinary, un-widened CANDIDATES resolution - below unchanged.""" + NAME_TO_RAW_VALUE_HISTORY (round 21, issue #1375) widens a dynamic + token to every DISTINCT value ever assigned to each name it + references, not just the single, possibly-stale one NAME_TO_RAW_ + VALUE's own order-blind collapse gives -- see `_assigned_raw_value_ + history`'s own docstring for the live bypass this originally closed. + + CRITICAL bug found by independent adversarial review (round 23, issue + #1375) and independently reproduced live: round 21's own widening was + scoped to a token that is EXACTLY one bare/braced whole-token + reference (`_BARE_OR_BRACED_VAR_REF_RE`) -- a token that fuses a + reference with other text (the ordinary `"$DIR/$FILE"` path-join + idiom, or `"${F}.bak"`) fell through to the ordinary, un-widened, + order-blind resolution below, unchanged. `DIR=sub; FILE=dirty.py; git + checkout -- "$DIR/$FILE"; DIR=other` resolved to `checkout_restore_ + paths=('other/dirty.py',)` -- a CONFIDENT, WRONG claim, not merely a + missed one: `$DIR` genuinely was `sub` at its actual point of use one + statement earlier. Confirmed live via a real bash proxy (stand-in + `git` binary on PATH, capturing its own argv) that real bash resolves + this to `checkout -- sub/dirty.py`, and end-to-end through the real + wrapper against a scratch repo with `sub/dirty.py` genuinely dirty: + the control command (no trailing reassignment) correctly denies + citing `sub/dirty.py`; the same command with the trailing `DIR=other` + wrongly ALLOWED (the live `git diff --quiet` check ran against the + wrong, nonexistent `other/dirty.py` and found it "clean"), while the + real, dirty file was never checked at all -- worse than the + already-accepted single-positional-ref Non-goal elsewhere in this + module, which at least makes NO claim rather than a wrong one (see + `_git_checkout_paths`'s own fourth-round paragraph for that same + "confident-wrong beats honest-silent" severity distinction). + + Closed by widening EVERY name referenced anywhere in a dynamic token + (not only a whole-token reference) that has more than one historical + value: `_multi_valued_names_referenced` finds those names, + `_bounded_history_combinations` builds the cartesian product of one + historical value per name (bounded by `_MAX_SUBSTITUTION_CANDIDATES`, + the same bound `_substitute_var_refs_candidates`'s own quote-boundary + expansion already uses), and `_substitute_var_refs_candidates` itself + -- unchanged, still the sole authority on quote/fusion-aware + resolution -- is called once per combination, with every combination's + own candidates unioned together. A name with only one historical + value (or none) contributes no combinations of its own, so a token + referencing no multiply-assigned name resolves in exactly one pass, + identical to before this fix.""" paths: list[str] = [] for tok in tokens: if not _is_dynamic(tok): paths.append(tok) continue - candidates = _substitute_var_refs_candidates(tok, name_to_raw_value, name_to_raw_value) + multi_valued = _multi_valued_names_referenced(tok, name_to_raw_value_history) + combinations = _bounded_history_combinations(multi_valued, name_to_raw_value_history) + if combinations is None: + return ( + f"a git checkout/restore command has a dynamic path argument ({tok!r}) with too many " + "historically-assigned readings to soundly enumerate -- an unresolved path cannot be safely " + "checked against the working tree, so this is denied outright", + (), + ) + candidates: list[str] = [] + for override in combinations: + variant = {**name_to_raw_value, **override} + variant_candidates = _substitute_var_refs_candidates(tok, variant, variant) + if not variant_candidates: + continue + candidates.extend(candidate for candidate in variant_candidates if candidate not in candidates) if not candidates or any(_is_dynamic(candidate) for candidate in candidates): return ( f"a git checkout/restore command has a dynamic path argument ({tok!r}) that could not be " @@ -3726,14 +3825,9 @@ def _resolve_path_tokens( "working tree, so this is denied outright", (), ) - bare_match = _BARE_OR_BRACED_VAR_REF_RE.fullmatch(tok) - history = name_to_raw_value_history.get(bare_match.group(1) or bare_match.group(2)) if bare_match else None - if history: - for value in history: - if value not in paths: - paths.append(value) - continue - paths.extend(candidates) + for candidate in candidates: + if candidate not in paths: + paths.append(candidate) return None, tuple(paths) diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index bc570be0..5c8f32ec 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -2073,6 +2073,44 @@ def test_restore_denied_when_a_path_argument_is_reassigned_after_use(tmp_path: P assert result.returncode == 2, f"stderr={result.stderr!r}" +def test_checkout_denied_when_a_fused_path_reference_is_reassigned_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-23 independent review, issue + #1375). Round 21's own history-widening fix was scoped to a token + that is EXACTLY one bare/braced whole-token reference -- the ordinary + `"$DIR/$FILE"` path-join idiom (a reference FUSED with a literal `/` + and another reference in the SAME token) fell through to the + ordinary, un-widened, order-blind resolution, producing a CONFIDENT, + WRONG `checkout_restore_paths` claim: `DIR=sub; FILE=dirty.py; git + checkout -- "$DIR/$FILE"; DIR=other` resolved to `('other/dirty.py',)` + alone, even though `$DIR` genuinely was `sub` at its actual point of + use one statement earlier -- so the live `git diff --quiet` check ran + against the wrong, nonexistent `other/dirty.py` and the genuinely + dirty `sub/dirty.py` was never checked at all.""" + repo_dir = tmp_path / "repo" + (repo_dir / "sub").mkdir(parents=True) + file_path = _init_repo_with_committed_file(repo_dir, filename="sub/dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run( + 'DIR=sub; FILE=dirty.py; git checkout -- "$DIR/$FILE"; DIR=other', + payload_cwd=str(repo_dir), + ) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_restore_denied_when_a_fused_path_reference_is_reassigned_after_use(tmp_path: Path) -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-23 finding was confirmed live for both subcommands.""" + repo_dir = tmp_path / "repo" + (repo_dir / "sub").mkdir(parents=True) + file_path = _init_repo_with_committed_file(repo_dir, filename="sub/dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run( + 'DIR=sub; FILE=dirty.py; git restore "$DIR/$FILE"; DIR=other', + payload_cwd=str(repo_dir), + ) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 72887cbd..ff07a6da 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -1976,18 +1976,83 @@ def test_resolve_path_tokens_history_widening_deduplicates_against_existing_path assert resolved == ("dirty.py", "other.py") -def test_resolve_path_tokens_does_not_widen_a_fused_reference() -> None: - """No false positive: a token that FUSES a reference with literal - text is NOT widened by the history mechanism (`_BARE_OR_BRACED_VAR_ - REF_RE` only matches a WHOLE-token bare/braced reference) -- it keeps - the ordinary, single-candidate resolution unchanged, the same - documented, deliberately narrower-than-full-soundness scoping this - fix uses everywhere else.""" +def test_resolve_path_tokens_widens_a_fused_reference() -> None: + """Regression pin for the real bypass found live by Step 8 + independent review, twenty-third round (issue #1375): round 21's own + widening was scoped to a token that is EXACTLY one bare/braced + whole-token reference, so a token that FUSES a reference with literal + text (the ordinary `"$DIR/$FILE"` path-join idiom, or `"${F}.py"` + here) fell through to the ordinary, un-widened, order-blind + resolution -- a CONFIDENT, WRONG single-candidate claim, not merely a + narrower-but-safe one. Every historical value for a name referenced + anywhere in the token -- fused or not -- is now widened.""" reason, resolved = checker._resolve_path_tokens(["${F}.py"], {"F": "dirty"}, {"F": ("dirty", "other")}) assert reason is None + assert resolved == ("dirty.py", "other.py") + + +def test_resolve_path_tokens_widens_two_fused_references_via_cartesian_product() -> None: + """The `"$DIR/$FILE"` path-join idiom itself, live-confirmed as this + round's own reproduction: two DIFFERENT names, each with its own + multi-valued history, both fused into the SAME token -- every + combination is widened, not just one name at a time.""" + reason, resolved = checker._resolve_path_tokens( + ["$DIR/$FILE"], + {"DIR": "other", "FILE": "dirty.py"}, + {"DIR": ("sub", "other"), "FILE": ("dirty.py",)}, + ) + assert reason is None + assert resolved == ("sub/dirty.py", "other/dirty.py") + + +def test_resolve_path_tokens_a_single_historical_value_widens_to_one_candidate_only() -> None: + """No false positive: a name with only ONE historical value (assigned + exactly once, or reassigned to the SAME value) contributes no extra + combinations -- the fused case degenerates to the same single + candidate the ordinary, un-widened resolution already gives.""" + reason, resolved = checker._resolve_path_tokens(["${F}.py"], {"F": "dirty"}, {"F": ("dirty",)}) + assert reason is None assert resolved == ("dirty.py",) +def test_bounded_history_combinations_returns_one_empty_combination_for_no_names() -> None: + """An empty NAMES set (no multi-valued name referenced) still yields + exactly one, no-op combination -- never `None` -- so a caller with + nothing to widen gets a single ordinary resolution pass, not a + spurious deny.""" + assert checker._bounded_history_combinations(set(), {}) == [{}] + + +def test_bounded_history_combinations_denies_when_the_product_is_too_large() -> None: + """Fail closed, matching `_substitute_var_refs_candidates`'s own + quote-boundary-expansion bound: a cartesian product exceeding + `_MAX_SUBSTITUTION_CANDIDATES` returns `None` rather than silently + enumerating only part of it.""" + history = { + "A": tuple(f"a{i}" for i in range(10)), + "B": tuple(f"b{i}" for i in range(10)), + } + assert checker._bounded_history_combinations({"A", "B"}, history) is None + + +def test_resolve_path_tokens_denies_when_the_combination_product_is_too_large() -> None: + """End-to-end (within `_resolve_path_tokens` itself, not just the + `_bounded_history_combinations` unit above): a token referencing two + names whose combined historical-value product exceeds `_MAX_ + SUBSTITUTION_CANDIDATES` is denied outright as unresolvable, rather + than silently enumerating only part of the combination space -- the + same fail-closed posture this module already takes for a + quote-boundary explosion.""" + history = { + "A": tuple(f"a{i}" for i in range(10)), + "B": tuple(f"b{i}" for i in range(10)), + } + reason, resolved = checker._resolve_path_tokens(["$A/$B"], {"A": "a0", "B": "b0"}, history) + assert reason is not None + assert "too many historically-assigned readings" in reason + assert resolved == () + + def test_classify_extracts_every_historical_path_when_a_checkout_path_is_reassigned_after_use() -> None: """End-to-end regression pin for the round-21 finding at the `classify()` level. Confirmed live before this fix: `F=dirty.py; git From 16b6fe4a4b94cc146c2b4cec4e9b6ce823de06f4 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 20:17:50 +0000 Subject: [PATCH 29/46] fix(hooks): close two more checkout/restore reassignment-ambiguity bugs A fresh, independent adversarial review of this PR's current head (round 24) found two more live, real-data-loss bugs in the checkout/ restore path-resolution machinery -- neither a re-occurrence of an already-fixed shape, both new instances of the same underlying order-blind reassignment-collapse family this PR has been closing since round 19. Finding 1: `_assigned_raw_values`/`_assigned_raw_value_history` both skip a dynamic-RHS assignment token entirely (`if _is_dynamic(token): continue`) -- correct in isolation, but this leaves a name's own EARLIER, static assignment on file completely untouched by a LATER, dynamic reassignment of the same name, silently trusted as still current. `DIR=sub; DIR=$(echo other); git checkout -- $DIR` resolved `checkout_restore_paths` to `('sub',)` -- confidently the WRONG, STALE value. Confirmed live via a real bash proxy (stand-in shell function) that real bash genuinely resolves `$DIR` to whatever the substitution evaluates to, not `sub`, at its actual point of use, and end-to-end through the real wrapper against a scratch repo with the genuinely dirty file: wrongly allowed with exit 0, and actually running the command afterward silently discarded the uncommitted edit. Finding 2: round 23's own `_multi_valued_names_referenced` (widening a dynamic token's referenced names to every historical value) only ever checked a `${!C}` indirect reference's FIRST-level name (C itself) for multi-valued history -- never the SECOND-level name C's own value actually points to, which is the one genuinely read at the point of use. `TARGET=sub; C=TARGET; git checkout -- ${!C}; TARGET=other` resolved to `('other',)` alone -- the wrong, order-blind-collapsed last value of TARGET, even though `${!C}` genuinely was `sub` at its actual point of use. Confirmed live the same way as Finding 1. Both fixes: - A new `_referenced_names` (renamed and extended from round 23's `_multi_valued_names_referenced`, which is now a thin filter over it) resolves an indirect reference two levels deep -- C itself, plus every name C's own current or historical value could itself name -- closing Finding 2. - A new `_names_with_dynamic_assignment(tokens)` detects every name with at least one dynamic-RHS assignment anywhere in the token stream. `_resolve_path_tokens` checks every name a dynamic token references against this set BEFORE attempting resolution and denies outright on a match -- the same posture this function already takes for a name with no recorded value at all, closing Finding 1. Threaded through the FULL recursive chain from the start (learning from round 20's own initial under-scoping, corrected in round 21): `classify()`, `_classify_tokens`, `_rule_command_substitution_content`, `_rule_array_literal_content`, `_rule_git_checkout_restore`, `_git_checkout_paths`, and `_git_restore_paths` all gained the new parameter, merged via UNION across outer/inner scope (the same convention `_merge_raw_value_histories` already uses) so a dynamic reassignment straddling a `$(...)` command-substitution boundary is covered too. Regression tests added at every established layer: end-to-end wrapper-level for both findings, both subcommands (hooks/test_gitapex_check_bash_safety.py), and unit/property level for `_referenced_names`'s two-level indirect resolution, `_names_with_ dynamic_assignment` (including a Hypothesis `@given` property test), and `_resolve_path_tokens`'s own poisoning check and false-positive guards, plus `classify()`-level end-to-end pins for both findings (tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI thresholds), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-24 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 239 ++++++++++++-- hooks/test_gitapex_check_bash_safety.py | 56 ++++ ...st_gitapex_check_bash_safety_properties.py | 307 ++++++++++++++---- 3 files changed, 502 insertions(+), 100 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index f92e413e..8c36847a 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -856,6 +856,7 @@ def _rule_command_substitution_content( name_to_raw_value_history: dict[str, tuple[str, ...]], name_to_value_write_biased: dict[str, str], name_to_raw_value_write_biased: dict[str, str], + names_with_dynamic_assignment: set[str], ) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `$(...)` command-substitution span's OWN inner content through this module's full rule set -- bash genuinely @@ -987,7 +988,13 @@ def _rule_command_substitution_content( same way round 21 found for the cd-biased case, still resolves correctly once the inner content reaches ITS OWN recursive `_classify_tokens`/`classify` call's own local `_rule_gh_api_write`/ - `_segment_loop_hit` invocations).""" + `_segment_loop_hit` invocations). + + NAMES_WITH_DYNAMIC_ASSIGNMENT (round 24, issue #1375) is threaded the + same way, so a name assigned a dynamic value inside a `$(...)` span + still poisons confidence in a checkout/restore path resolved from + that same name OUTSIDE the span (or vice versa) -- see `_names_with_ + dynamic_assignment`'s own docstring for the live bypass it closes.""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -1024,6 +1031,7 @@ def _rule_command_substitution_content( name_to_raw_value_history, name_to_value_write_biased, name_to_raw_value_write_biased, + names_with_dynamic_assignment, ) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) @@ -1047,6 +1055,7 @@ def _rule_command_substitution_content( name_to_raw_value_history, name_to_value_write_biased, name_to_raw_value_write_biased, + names_with_dynamic_assignment, ) is_git_push = is_git_push or inner_verdict.is_git_push checkout_restore_paths.extend(inner_verdict.checkout_restore_paths) @@ -2635,6 +2644,7 @@ def _rule_array_literal_content( name_to_raw_value_history: dict[str, tuple[str, ...]], name_to_value_write_biased: dict[str, str], name_to_raw_value_write_biased: dict[str, str], + names_with_dynamic_assignment: set[str], ) -> tuple[str | None, bool, tuple[str, ...]]: """Recursively classify each `NAME=(...)` array-literal span's OWN inner content through this module's full rule set -- bash genuinely @@ -2753,7 +2763,9 @@ def _rule_array_literal_content( substitution_content`'s own identical parameters' docstring paragraph for what they mean and why they are threaded here even though this function's own recursive call site does not itself invoke - `_rule_gh_api_write`/`_segment_loop_hit` directly.""" + `_rule_gh_api_write`/`_segment_loop_hit` directly. NAMES_WITH_ + DYNAMIC_ASSIGNMENT (round 24, issue #1375) is threaded the same way + -- see `_names_with_dynamic_assignment`'s own docstring.""" is_git_push = False checkout_restore_paths: list[str] = [] i = 0 @@ -2779,6 +2791,7 @@ def _rule_array_literal_content( name_to_raw_value_history, name_to_value_write_biased, name_to_raw_value_write_biased, + names_with_dynamic_assignment, ) is_git_push = is_git_push or reading_verdict.is_git_push checkout_restore_paths.extend(reading_verdict.checkout_restore_paths) @@ -3670,29 +3683,116 @@ def _is_git_push_segment(seg: list[str], name_to_raw_value: dict[str, str]) -> b _CHECKOUT_BRANCH_CREATION_FLAGS = {"-b", "-B", "--orphan"} -def _multi_valued_names_referenced(token: str, name_to_raw_value_history: dict[str, tuple[str, ...]]) -> set[str]: +def _referenced_names( + token: str, name_to_raw_value: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]] +) -> set[str]: """Every variable NAME referenced anywhere in TOKEN -- braced - (`${NAME}`), default-clause (`${NAME:-x}`), indirect (`${!NAME}`), or + (`${NAME}`), default-clause (`${NAME:-x}`), indirect (`${!NAME}`, + TWO levels: NAME itself, PLUS every name NAME's own current or + historical value could itself name -- see the CRITICAL-bug paragraph + below for why the second level is the one that actually matters), or unbraced (`$NAME`, including every non-empty PREFIX of an unbraced run, mirroring `_unbraced_ref_options`'s own prefix enumeration, - since an unbraced reference is ambiguous about where the name ends) - -- whose own NAME_TO_RAW_VALUE_HISTORY entry carries MORE than one - distinct value. A name assigned only once (or never) is excluded: its - single reading already matches the ordinary, un-widened resolution, - so there is nothing to widen. Used by `_resolve_path_tokens`'s own - fused-reference history widening (see that function's own docstring) - to find which names' history actually needs enumerating, not to - resolve the token itself -- `_substitute_var_refs_candidates` still - does the real, quote/fusion-aware resolution, once per combination.""" + since an unbraced reference is ambiguous about where the name ends). + + CRITICAL bug found by independent adversarial review (round 24, issue + #1375) and independently reproduced live, in this function's own + prior form (then named `_multi_valued_names_referenced`, collecting + only names with multi-valued history directly): for an indirect + `${!C}` reference, only C ITSELF was ever checked for multi-valued + history -- never the SECOND-level name C's own value actually points + to, which is the one `_substitute_var_refs_candidates`'s own + indirect-reference resolution (see that function's own docstring) + genuinely reads at the point of use. `TARGET=sub; C=TARGET; git + checkout -- ${!C}; TARGET=other` -- `${!C}` resolves through C (which + has only ONE historical value, "TARGET", so C itself was never a + widening candidate) to TARGET, which DOES have two historical values + ("sub" then "other") -- but TARGET was never added to the widening + set at all, so `checkout_restore_paths` resolved to `('other',)` + alone, the WRONG, order-blind-collapsed last value of TARGET (real + bash: `${!C}` genuinely was `sub` at its actual point of use). + Confirmed live end-to-end through the real wrapper against a scratch + repo with `sub` genuinely dirty and `other` clean: wrongly allowed + with exit 0, and actually running the command afterward silently + discarded the uncommitted edit to `sub`. Closed by treating EVERY + name C's own current value (NAME_TO_RAW_VALUE) or any historical + value (NAME_TO_RAW_VALUE_HISTORY) could itself name as a referenced + name in its own right, not just C.""" names: set[str] = set() for match in _VAR_REF_FULL_RE.finditer(token): braced_name, default_name, _default_text, indirect_name, unbraced_run = match.groups() - candidate_names = [n for n in (braced_name, default_name, indirect_name) if n is not None] - if unbraced_run is not None: - candidate_names.extend(unbraced_run[:i] for i in range(len(unbraced_run), 0, -1)) - for name in candidate_names: - if len(name_to_raw_value_history.get(name, ())) > 1: + for name in (braced_name, default_name): + if name is not None: names.add(name) + if indirect_name is not None: + names.add(indirect_name) + second_level = set(name_to_raw_value_history.get(indirect_name, ())) + current = name_to_raw_value.get(indirect_name) + if current is not None: + second_level.add(current) + names.update(second_level) + if unbraced_run is not None: + names.update(unbraced_run[:i] for i in range(len(unbraced_run), 0, -1)) + return names + + +def _multi_valued_names_referenced( + token: str, name_to_raw_value: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]] +) -> set[str]: + """Every name `_referenced_names` finds in TOKEN whose own NAME_TO_ + RAW_VALUE_HISTORY entry carries MORE than one distinct value. A name + assigned only once (or never) is excluded: its single reading already + matches the ordinary, un-widened resolution, so there is nothing to + widen. Used by `_resolve_path_tokens`'s own fused-reference history + widening (see that function's own docstring) to find which names' + history actually needs enumerating, not to resolve the token itself + -- `_substitute_var_refs_candidates` still does the real, quote/ + fusion-aware resolution, once per combination.""" + return { + name + for name in _referenced_names(token, name_to_raw_value, name_to_raw_value_history) + if len(name_to_raw_value_history.get(name, ())) > 1 + } + + +def _names_with_dynamic_assignment(tokens: list[str]) -> set[str]: + """Every NAME assigned a DYNAMIC value (containing `$`/backtick) + anywhere in TOKENS. + + CRITICAL bug found by independent adversarial review (round 24, issue + #1375) and independently reproduced live: `_assigned_raw_values`/ + `_assigned_raw_value_history` both skip a dynamic-RHS assignment + token entirely (`if _is_dynamic(token): continue`) -- correct in + isolation (this classifier genuinely cannot know what a `$(...)` + substitution will evaluate to), but this means a name's own EARLIER, + STATIC assignment stays in NAME_TO_RAW_VALUE/NAME_TO_RAW_VALUE_ + HISTORY completely untouched, silently trusted as still current, even + though a LATER, dynamic reassignment of the SAME name means its + actual value at the point of use is genuinely unknown. `DIR=sub; + DIR=$(echo other); git checkout -- $DIR` resolved `checkout_restore_ + paths` to `('sub',)` -- confidently the WRONG, STALE value (real + bash: `$DIR` genuinely was `other`, whatever the substitution + evaluates to, at its actual point of use). Confirmed live end-to-end + through the real wrapper against a scratch repo with `other` + genuinely dirty and `sub` clean: wrongly allowed with exit 0, and + actually running the command afterward silently discarded the + uncommitted edit to `other`. Also reproduced identically for the + fused-reference form (`"$DIR/f"`). + + Closed by `_resolve_path_tokens` treating a name in this set as + forced-unresolvable (deny outright) wherever referenced -- the same + posture this module already takes for a name that was NEVER assigned + any value at all (a name with a static-only history, or no history, + resolves exactly as before; a name with even one dynamic assignment + anywhere loses all confidence, matching how this classifier already + treats a name with NO recorded static value).""" + names: set[str] = set() + for token in tokens: + if not _is_dynamic(token): + continue + match = _ASSIGN_RE.match(token) + if match: + names.add(match.group(1)) return names @@ -3720,7 +3820,10 @@ def _bounded_history_combinations( def _resolve_path_tokens( - tokens: list[str], name_to_raw_value: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]] + tokens: list[str], + name_to_raw_value: dict[str, str], + name_to_raw_value_history: dict[str, tuple[str, ...]], + names_with_dynamic_assignment: set[str], ) -> tuple[str | None, tuple[str, ...]]: """Resolve every token in TOKENS to one or more literal path candidates for a `git checkout`/`git restore` invocation. A literal @@ -3796,13 +3899,38 @@ def _resolve_path_tokens( own candidates unioned together. A name with only one historical value (or none) contributes no combinations of its own, so a token referencing no multiply-assigned name resolves in exactly one pass, - identical to before this fix.""" + identical to before this fix. + + NAMES_WITH_DYNAMIC_ASSIGNMENT (round 24, issue #1375) closes a + DIFFERENT bug in the same family, found live independently of the + fused-reference fix above: `_assigned_raw_values`/`_assigned_raw_ + value_history` both build NAME_TO_RAW_VALUE/NAME_TO_RAW_VALUE_HISTORY + by skipping a dynamic-RHS assignment token entirely, so a name's + EARLIER, static assignment stays on file completely untouched by a + LATER, dynamic reassignment of the SAME name -- silently trusted as + still current, even though the real value at the point of use is now + genuinely unknown. See `_names_with_dynamic_assignment`'s own + docstring for the live reproduction. Every name a dynamic token + references (via `_referenced_names`, the same two-level-indirect- + aware extraction `_multi_valued_names_referenced` itself now uses) is + checked against this set BEFORE any resolution is attempted; a match + denies outright, the same posture this function already takes for a + name with no recorded value at all -- a dynamic reassignment does not + get to silently fall back to a stale earlier reading.""" paths: list[str] = [] for tok in tokens: if not _is_dynamic(tok): paths.append(tok) continue - multi_valued = _multi_valued_names_referenced(tok, name_to_raw_value_history) + referenced = _referenced_names(tok, name_to_raw_value, name_to_raw_value_history) + if referenced & names_with_dynamic_assignment: + return ( + f"a git checkout/restore command has a dynamic path argument ({tok!r}) that references a name " + "also assigned a dynamically-constructed value elsewhere in the command -- that name's actual " + "value at the point of use cannot be soundly trusted, so this is denied outright", + (), + ) + multi_valued = _multi_valued_names_referenced(tok, name_to_raw_value, name_to_raw_value_history) combinations = _bounded_history_combinations(multi_valued, name_to_raw_value_history) if combinations is None: return ( @@ -3835,6 +3963,7 @@ def _git_checkout_paths( tokens_after: list[str], name_to_raw_value: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]], + names_with_dynamic_assignment: set[str], ) -> tuple[str | None, tuple[str, ...]]: """checkout_restore_paths for a `git checkout` invocation, TOKENS_AFTER being every segment token following the literal `checkout` word. @@ -3955,12 +4084,16 @@ def _git_checkout_paths( "could append paths at runtime this classifier cannot see, so this is denied outright", (), ) - return _resolve_path_tokens(after, name_to_raw_value, name_to_raw_value_history) + return _resolve_path_tokens(after, name_to_raw_value, name_to_raw_value_history, names_with_dynamic_assignment) positionals = [t for t in tokens_after if not t.startswith("-")] if len(positionals) >= 2: - return _resolve_path_tokens(positionals, name_to_raw_value, name_to_raw_value_history) + return _resolve_path_tokens( + positionals, name_to_raw_value, name_to_raw_value_history, names_with_dynamic_assignment + ) if len(positionals) == 1 and not _is_dynamic(positionals[0]) and positionals[0] in (".", ".."): - return _resolve_path_tokens(positionals, name_to_raw_value, name_to_raw_value_history) + return _resolve_path_tokens( + positionals, name_to_raw_value, name_to_raw_value_history, names_with_dynamic_assignment + ) return None, () @@ -3968,6 +4101,7 @@ def _git_restore_paths( tokens_after: list[str], name_to_raw_value: dict[str, str], name_to_raw_value_history: dict[str, tuple[str, ...]], + names_with_dynamic_assignment: set[str], ) -> tuple[str | None, tuple[str, ...]]: """checkout_restore_paths for a `git restore` invocation, TOKENS_AFTER being every segment token following the literal `restore` word. @@ -4061,7 +4195,9 @@ def _git_restore_paths( i += 1 if saw_staged and not saw_worktree: return None, () - return _resolve_path_tokens(path_tokens, name_to_raw_value, name_to_raw_value_history) + return _resolve_path_tokens( + path_tokens, name_to_raw_value, name_to_raw_value_history, names_with_dynamic_assignment + ) # A whole token that is EXACTLY one `${NAME-}`/`${NAME:-}` (empty default @@ -4628,6 +4764,7 @@ def _rule_git_checkout_restore( raw_assigned_git_biased: dict[str, str], raw_assigned_cd_biased: dict[str, str], raw_assigned_history: dict[str, tuple[str, ...]], + names_with_dynamic_assignment: set[str], ) -> tuple[str | None, tuple[str, ...]]: """Extract every `checkout_restore_paths` candidate across every segment of one command, denying outright on any segment where this @@ -4720,7 +4857,10 @@ def _rule_git_checkout_restore( paragraph for the live bypass this one closes. RAW_ASSIGNED_HISTORY is passed straight through into `_git_checkout_paths`/`_git_restore_ paths` (round 21, issue #1375) -- see `_assigned_raw_value_history`'s - own docstring for what it means and the live bypass it closes.""" + own docstring for what it means and the live bypass it closes. + NAMES_WITH_DYNAMIC_ASSIGNMENT (round 24, issue #1375) is threaded the + same way into both -- see `_names_with_dynamic_assignment`'s own + docstring for what it means and the live bypass it closes.""" saw_cd = False all_paths: list[str] = [] for seg in segments: @@ -4748,9 +4888,13 @@ def _rule_git_checkout_restore( (), ) if subcommand == "checkout": - deny_reason, paths = _git_checkout_paths(tokens_after, raw_assigned, raw_assigned_history) + deny_reason, paths = _git_checkout_paths( + tokens_after, raw_assigned, raw_assigned_history, names_with_dynamic_assignment + ) else: - deny_reason, paths = _git_restore_paths(tokens_after, raw_assigned, raw_assigned_history) + deny_reason, paths = _git_restore_paths( + tokens_after, raw_assigned, raw_assigned_history, names_with_dynamic_assignment + ) if deny_reason: return deny_reason, () all_paths.extend(paths) @@ -4961,6 +5105,7 @@ def classify( outer_name_to_raw_value_history: dict[str, tuple[str, ...]] | None = None, outer_name_to_value_write_biased: dict[str, str] | None = None, outer_name_to_raw_value_write_biased: dict[str, str] | None = None, + outer_names_with_dynamic_assignment: set[str] | None = None, ) -> Verdict: """Classify one Bash tool_input.command string. Fails closed (deny) on anything shlex cannot tokenize -- an unparseable command is exactly the @@ -4979,12 +5124,14 @@ def classify( OUTER_NAME_TO_RAW_VALUE_GIT_BIASED (round 19, issue #1375), OUTER_NAME_TO_RAW_VALUE_CD_BIASED and OUTER_NAME_TO_RAW_VALUE_HISTORY - (round 21, issue #1375), and OUTER_NAME_TO_VALUE_WRITE_BIASED/ - OUTER_NAME_TO_RAW_VALUE_WRITE_BIASED (round 22, issue #1375) are the + (round 21, issue #1375), OUTER_NAME_TO_VALUE_WRITE_BIASED/ + OUTER_NAME_TO_RAW_VALUE_WRITE_BIASED (round 22, issue #1375), and + OUTER_NAMES_WITH_DYNAMIC_ASSIGNMENT (round 24, issue #1375) are the same recursive call's own analogous further arguments -- see `_classify_tokens`'s own docstring and `_find_git_checkout_restore`'s/ - `_assigned_raw_value_history`'s/`_assigned_literals_biased_toward`'s - own docstrings for what each means and the live bypass each closes.""" + `_assigned_raw_value_history`'s/`_assigned_literals_biased_toward`'s/ + `_names_with_dynamic_assignment`'s own docstrings for what each means + and the live bypass each closes.""" try: tokens = tokenize(command) except TokenizeError as error: @@ -4998,6 +5145,7 @@ def classify( outer_name_to_raw_value_history, outer_name_to_value_write_biased, outer_name_to_raw_value_write_biased, + outer_names_with_dynamic_assignment, ) @@ -5010,6 +5158,7 @@ def _classify_tokens( outer_name_to_raw_value_history: dict[str, tuple[str, ...]] | None = None, outer_name_to_value_write_biased: dict[str, str] | None = None, outer_name_to_raw_value_write_biased: dict[str, str] | None = None, + outer_names_with_dynamic_assignment: set[str] | None = None, ) -> Verdict: """The token-level core of `classify` -- split out so `_rule_command_ substitution_content` can recurse into a `$(...)` span's own inner @@ -5080,7 +5229,19 @@ def _classify_tokens( write` and `_segment_loop_hit` below, alongside the ordinary ASSIGNED/ RAW_ASSIGNED attempt -- see `_assigned_literals_biased_toward`'s own docstring for the live bypass this closes on those two HARD-DENY - consumers specifically.""" + consumers specifically. + + OUTER_NAMES_WITH_DYNAMIC_ASSIGNMENT (round 24, issue #1375) is a + further, parallel outer-scope argument -- a UNION merge (`outer | + _names_with_dynamic_assignment(tokens)`, the same union convention + OUTER_NAME_TO_RAW_VALUE_HISTORY already uses, not the plain + `{**outer, **inner}` shadowing every dict-shaped pair above uses), + threaded the same way into the same two recursive calls plus `_rule_ + git_checkout_restore`'s own sixth argument. Unlike the write-biased + pair above, this one IS threaded into `_rule_git_checkout_restore` + (this bug lives in checkout/restore path resolution specifically) -- + see `_names_with_dynamic_assignment`'s own docstring for the live + bypass this closes.""" outer_literals = outer_name_to_value or {} outer_raw = outer_name_to_raw_value or {} outer_raw_git_biased = outer_name_to_raw_value_git_biased or {} @@ -5088,6 +5249,7 @@ def _classify_tokens( outer_raw_history = outer_name_to_raw_value_history or {} outer_write_biased = outer_name_to_value_write_biased or {} outer_raw_write_biased = outer_name_to_raw_value_write_biased or {} + outer_dynamic_names = outer_names_with_dynamic_assignment or set() merged_name_to_value = {**outer_literals, **_assigned_literals(tokens)} merged_name_to_raw_value = {**outer_raw, **_assigned_raw_values(tokens)} merged_name_to_raw_value_git_biased = { @@ -5109,6 +5271,7 @@ def _classify_tokens( **outer_raw_write_biased, **_assigned_raw_values_biased_toward(tokens, _WATCHED_WRITE_BIAS), } + merged_names_with_dynamic_assignment = outer_dynamic_names | _names_with_dynamic_assignment(tokens) content_reason, content_is_git_push, content_checkout_restore_paths = _rule_command_substitution_content( tokens, @@ -5119,6 +5282,7 @@ def _classify_tokens( merged_name_to_raw_value_history, merged_name_to_value_write_biased, merged_name_to_raw_value_write_biased, + merged_names_with_dynamic_assignment, ) if content_reason: return Verdict(True, content_reason, content_is_git_push, content_checkout_restore_paths) @@ -5132,6 +5296,7 @@ def _classify_tokens( merged_name_to_raw_value_history, merged_name_to_value_write_biased, merged_name_to_raw_value_write_biased, + merged_names_with_dynamic_assignment, ) is_git_push = content_is_git_push or array_content_is_git_push checkout_restore_paths = content_checkout_restore_paths + array_content_checkout_restore_paths @@ -5159,6 +5324,7 @@ def _classify_tokens( **outer_raw_write_biased, **_assigned_raw_values_biased_toward(tokens, _WATCHED_WRITE_BIAS), } + names_with_dynamic_assignment = outer_dynamic_names | _names_with_dynamic_assignment(tokens) lowered_command = " ".join(tokens).lower() is_git_push = is_git_push or any(_is_git_push_segment(seg, raw_assigned) for seg in segments) @@ -5203,7 +5369,12 @@ def _classify_tokens( ) own_checkout_restore_hit, own_checkout_restore_paths = _rule_git_checkout_restore( - segments, raw_assigned, raw_assigned_git_biased, raw_assigned_cd_biased, raw_assigned_history + segments, + raw_assigned, + raw_assigned_git_biased, + raw_assigned_cd_biased, + raw_assigned_history, + names_with_dynamic_assignment, ) checkout_restore_paths = checkout_restore_paths + own_checkout_restore_paths if own_checkout_restore_hit: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 5c8f32ec..27f66d79 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -2111,6 +2111,62 @@ def test_restore_denied_when_a_fused_path_reference_is_reassigned_after_use(tmp_ assert result.returncode == 2, f"stderr={result.stderr!r}" +def test_checkout_denied_when_the_path_name_is_reassigned_to_a_dynamic_value_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-24 independent review, issue + #1375). `_assigned_raw_values`/`_assigned_raw_value_history` both skip + a dynamic-RHS assignment token entirely, so a name's own EARLIER, + static assignment stayed on file untouched by a LATER, dynamic + reassignment of the SAME name -- silently trusted as still current. + Live-verified before this fix: `DIR=sub; DIR=$(echo other); git + checkout -- $DIR` resolved `checkout_restore_paths` to `('sub',)` -- + confidently the WRONG, STALE value, since real bash genuinely + resolves `$DIR` to whatever the substitution evaluates to at its + actual point of use, not `sub`.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("DIR=dirty.py; DIR=$(echo other.py); git checkout -- $DIR", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_restore_denied_when_the_path_name_is_reassigned_to_a_dynamic_value_after_use(tmp_path: Path) -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-24 finding was confirmed live for both subcommands.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("DIR=dirty.py; DIR=$(echo other.py); git restore $DIR", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_checkout_denied_when_an_indirect_reference_target_is_reassigned_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-24 independent review, issue + #1375), second class: round 23's own `_multi_valued_names_referenced` + widened only the FIRST-level name of a `${!NAME}` indirect reference, + never the SECOND-level target it actually points to at the point of + use. Live-verified before this fix: `TARGET=dirty.py; C=TARGET; git + checkout -- ${!C}; TARGET=other.py` resolved `checkout_restore_paths` + to `('other.py',)` alone -- the WRONG, order-blind-collapsed last + value of TARGET, since real bash genuinely resolves `${!C}` to + `dirty.py` at its actual point of use.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("TARGET=dirty.py; C=TARGET; git checkout -- ${!C}; TARGET=other.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_restore_denied_when_an_indirect_reference_target_is_reassigned_after_use(tmp_path: Path) -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-24 finding's second class was confirmed live for both + subcommands.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("TARGET=dirty.py; C=TARGET; git restore ${!C}; TARGET=other.py", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index ff07a6da..a2894b5d 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -1358,7 +1358,7 @@ def test_rule_command_substitution_content_detects_an_embedded_install(tool: str a punctuation character shlex breaks a word at, so an assignment's `NAME=` prefix stays fused onto the leading `$` in the same token.""" tokens = ["x=$", "(", tool, "install", "evil-pkg", ")"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None @@ -1374,7 +1374,7 @@ def test_rule_command_substitution_content_allows_harmless_inner_content(value: silently dropping a non-denying inner `is_git_push=True` signal (see the function's own docstring).""" tokens = ["echo", "$", "(", "date", value, ")"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) == (None, False, ()) # --- Issue #1326 Stage 1, fifteenth round: bash's own leading-assignment ---- @@ -1493,7 +1493,7 @@ def test_rule_array_literal_content_detects_a_denied_pair_regardless_of_a_leadin `Y=1; A=(uv install $Y); "${A[@]}"` was wrongly ALLOWED before this function existed.""" tokens = ["dummy=", "(", f"${first}", "uv", "install", f"${second}", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None @@ -1512,7 +1512,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_bare_ref(unse fused with other text (not a bare whole-token reference), must NOT be collapsed -- that shape does not word-split away to nothing.""" tokens = ["dummy=", "(", f"${unset_name}", verb_a, "install", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None @@ -1532,13 +1532,13 @@ def test_rule_array_literal_content_allows_harmless_content() -> None: denied pattern, with or without a leading unassigned reference, stays allowed.""" tokens = ["dummy=", "(", "$NEVERSET", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) == (None, False, ()) def test_rule_array_literal_content_no_span_present() -> None: """Robustness: a token stream with no array-literal span at all (e.g. an ordinary command) returns cleanly, never a crash.""" - assert checker._rule_array_literal_content(["echo", "hi"], {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(["echo", "hi"], {}, {}, {}, {}, {}, {}, {}, set()) == (None, False, ()) def test_strip_leading_unassigned_bare_refs_stops_at_a_fused_token() -> None: @@ -1564,7 +1564,7 @@ def test_rule_array_literal_content_empty_array_is_harmless() -> None: """No false positive / no crash: an empty array literal `NAME=()` has no inner content to recursively classify at all.""" tokens = ["dummy=", "(", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) == (None, False, ()) def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leading_unassigned_ref() -> None: @@ -1573,7 +1573,7 @@ def test_rule_array_literal_content_skips_the_collapsed_reading_without_a_leadin `_strip_leading_unassigned_bare_refs` to strip -- the collapsed reading equals the as-is one, so only one classification is needed.""" tokens = ["dummy=", "(", "echo", "harmless", ")"] - assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) == (None, False, ()) def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> None: @@ -1590,7 +1590,7 @@ def test_rule_array_literal_content_denies_only_on_the_collapsed_reading() -> No real, with a dynamic verb argument right after it -- exactly B2's own watched shape.""" tokens = ["dummy=", "(", "$NEVERSET", "uv", "$VERB", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None assert "unassigned reference" in reason @@ -1632,7 +1632,7 @@ def test_rule_array_literal_content_collapses_a_leading_unassigned_braced_bare_r this round, silently degrading the collapsed reading to a no-op for this shape.""" tokens = ["dummy=", "(", f"${{{unset_name}}}", verb_a, "install", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None @@ -1647,7 +1647,7 @@ def test_rule_array_literal_content_detects_an_outer_scope_resolved_pair() -> No recursive `_classify_tokens` call.""" tokens = ["dummy=", "(", "$G", "$P", "$M", ")"] outer = {"G": "gh", "P": "pr", "M": "merge"} - reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer, outer, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, outer, outer, outer, {}, {}, {}, {}, set()) assert reason is not None @@ -1684,7 +1684,7 @@ def test_rule_command_substitution_content_detects_an_outer_scope_resolved_check tokens = ["x=$", "(", "$G", "checkout", "--", "dirty.py", ")"] outer = {"G": "git"} reason, _, checkout_restore_paths = checker._rule_command_substitution_content( - tokens, outer, outer, outer, {}, {}, {}, {} + tokens, outer, outer, outer, {}, {}, {}, {}, set() ) assert reason is None assert checkout_restore_paths == ("dirty.py",) @@ -1848,7 +1848,7 @@ def test_rule_git_checkout_restore_recognizes_a_cd_biased_reassigned_relocator() some point, so the earlier relocation is still flagged and the checkout is denied rather than confidently, wrongly resolved.""" segments = [["$X", "sub"], ["git", "checkout", "--", "dirty.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "elsewhere"}, {}, {"X": "cd"}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "elsewhere"}, {}, {"X": "cd"}, {}, set()) assert reason is not None assert "cd" in reason or "pushd" in reason or "popd" in reason assert resolved == () @@ -1960,7 +1960,7 @@ def test_resolve_path_tokens_widens_a_bare_reference_to_its_full_history() -> No multiple distinct historical values now extracts ALL of them as separate candidates, not just the single, possibly-stale one the collapsed NAME_TO_RAW_VALUE dict gives.""" - reason, resolved = checker._resolve_path_tokens(["$F"], {"F": "other.py"}, {"F": ("dirty.py", "other.py")}) + reason, resolved = checker._resolve_path_tokens(["$F"], {"F": "other.py"}, {"F": ("dirty.py", "other.py")}, set()) assert reason is None assert resolved == ("dirty.py", "other.py") @@ -1970,7 +1970,7 @@ def test_resolve_path_tokens_history_widening_deduplicates_against_existing_path from an earlier, literal token in the same command is not appended twice.""" reason, resolved = checker._resolve_path_tokens( - ["dirty.py", "$F"], {"F": "other.py"}, {"F": ("dirty.py", "other.py")} + ["dirty.py", "$F"], {"F": "other.py"}, {"F": ("dirty.py", "other.py")}, set() ) assert reason is None assert resolved == ("dirty.py", "other.py") @@ -1986,7 +1986,7 @@ def test_resolve_path_tokens_widens_a_fused_reference() -> None: resolution -- a CONFIDENT, WRONG single-candidate claim, not merely a narrower-but-safe one. Every historical value for a name referenced anywhere in the token -- fused or not -- is now widened.""" - reason, resolved = checker._resolve_path_tokens(["${F}.py"], {"F": "dirty"}, {"F": ("dirty", "other")}) + reason, resolved = checker._resolve_path_tokens(["${F}.py"], {"F": "dirty"}, {"F": ("dirty", "other")}, set()) assert reason is None assert resolved == ("dirty.py", "other.py") @@ -2000,6 +2000,7 @@ def test_resolve_path_tokens_widens_two_fused_references_via_cartesian_product() ["$DIR/$FILE"], {"DIR": "other", "FILE": "dirty.py"}, {"DIR": ("sub", "other"), "FILE": ("dirty.py",)}, + set(), ) assert reason is None assert resolved == ("sub/dirty.py", "other/dirty.py") @@ -2010,7 +2011,7 @@ def test_resolve_path_tokens_a_single_historical_value_widens_to_one_candidate_o exactly once, or reassigned to the SAME value) contributes no extra combinations -- the fused case degenerates to the same single candidate the ordinary, un-widened resolution already gives.""" - reason, resolved = checker._resolve_path_tokens(["${F}.py"], {"F": "dirty"}, {"F": ("dirty",)}) + reason, resolved = checker._resolve_path_tokens(["${F}.py"], {"F": "dirty"}, {"F": ("dirty",)}, set()) assert reason is None assert resolved == ("dirty.py",) @@ -2047,12 +2048,120 @@ def test_resolve_path_tokens_denies_when_the_combination_product_is_too_large() "A": tuple(f"a{i}" for i in range(10)), "B": tuple(f"b{i}" for i in range(10)), } - reason, resolved = checker._resolve_path_tokens(["$A/$B"], {"A": "a0", "B": "b0"}, history) + reason, resolved = checker._resolve_path_tokens(["$A/$B"], {"A": "a0", "B": "b0"}, history, set()) assert reason is not None assert "too many historically-assigned readings" in reason assert resolved == () +def test_referenced_names_resolves_an_indirect_reference_to_its_second_level_target() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-fourth round (issue #1375): `${!C}` is a TWO-level + reference (C's own value names a SECOND variable), but the prior + version of this extraction only ever checked C itself for multi- + valued history -- never the second-level name C's value actually + points to, which is the one genuinely read at the point of use. Both + C itself and the name it currently points to must be in the + referenced set.""" + names = checker._referenced_names("${!C}", {"C": "TARGET"}, {}) + assert names == {"C", "TARGET"} + + +def test_referenced_names_resolves_every_historical_second_level_target() -> None: + """Companion to the pin above: C's own HISTORY (not just its current + reading) is also searched for second-level target names, since C + itself may have been reassigned to point somewhere else at an + earlier point.""" + names = checker._referenced_names("${!C}", {"C": "OTHER"}, {"C": ("TARGET", "OTHER")}) + assert names == {"C", "TARGET", "OTHER"} + + +def test_referenced_names_resolves_a_second_level_target_with_no_current_reading() -> None: + """C's own history is searched even when C has no CURRENT reading at + all in NAME_TO_RAW_VALUE (e.g. C was only ever assigned inside a + scope this dict no longer reflects) -- only C's history need supply + the second-level name.""" + names = checker._referenced_names("${!C}", {}, {"C": ("TARGET",)}) + assert names == {"C", "TARGET"} + + +def test_multi_valued_names_referenced_widens_a_second_level_indirect_target() -> None: + """End-to-end for `_multi_valued_names_referenced` itself: TARGET (the + second-level name `${!C}` actually resolves through) is included when + ITS OWN history is multi-valued, even though C's own history has only + one entry.""" + history = {"C": ("TARGET",), "TARGET": ("sub", "other")} + assert checker._multi_valued_names_referenced("${!C}", {"C": "TARGET"}, history) == {"TARGET"} + + +def test_resolve_path_tokens_widens_an_indirect_reference_second_level_target() -> None: + """End-to-end regression pin for the round-24 finding at the + `_resolve_path_tokens` level. Confirmed live before this fix: + `TARGET=sub; C=TARGET; git checkout -- ${!C}; TARGET=other` resolved + `checkout_restore_paths` to `('other',)` alone -- the WRONG, + order-blind-collapsed last value of TARGET (real bash: `${!C}` + genuinely was `sub` at its actual point of use).""" + reason, resolved = checker._resolve_path_tokens( + ["${!C}"], {"C": "TARGET", "TARGET": "other"}, {"TARGET": ("sub", "other")}, set() + ) + assert reason is None + assert resolved == ("sub", "other") + + +def test_names_with_dynamic_assignment_finds_a_dynamically_reassigned_name() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-fourth round (issue #1375): `_assigned_raw_values`/ + `_assigned_raw_value_history` both skip a dynamic-RHS assignment + token entirely, so a name's own earlier static assignment stays on + file untouched by a later dynamic reassignment -- this function is + the dedicated detector that closes that gap.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", "DIR=$(echo other)"]) == {"DIR"} + + +def test_names_with_dynamic_assignment_ignores_a_purely_static_name() -> None: + """No false positive: a name assigned only static values anywhere is + not flagged.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", "DIR=other"]) == set() + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES) +def test_names_with_dynamic_assignment_matches_model_for_a_static_then_dynamic_reassignment( + name: str, static_value: str, dynamic_value: str +) -> None: + """Model-based: for ANY identifier assigned a static value and then + reassigned a dynamic one, `_names_with_dynamic_assignment` always + includes it -- and a second, entirely unrelated identifier that is + only ever assigned statically is never included alongside it.""" + other_name = name + "_OTHER" + tokens = [f"{name}={static_value}", f"{name}=$({dynamic_value})", f"{other_name}={static_value}"] + result = checker._names_with_dynamic_assignment(tokens) + assert name in result + assert other_name not in result + + +def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: + """End-to-end regression pin for the round-24 finding at the + `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= + sub; DIR=$(echo other); git checkout -- $DIR` resolved `checkout_ + restore_paths` to `('sub',)` -- confidently the WRONG, STALE value + (real bash: `$DIR` genuinely was whatever the substitution evaluated + to, not `sub`, at its actual point of use).""" + reason, resolved = checker._resolve_path_tokens(["$DIR"], {"DIR": "sub"}, {}, {"DIR"}) + assert reason is not None + assert "also assigned a dynamically-constructed value" in reason + assert resolved == () + + +def test_resolve_path_tokens_ignores_a_dynamic_assignment_to_an_unrelated_name() -> None: + """No false positive: NAMES_WITH_DYNAMIC_ASSIGNMENT poisons only the + SPECIFIC name it names -- a dynamic reassignment of a completely + unrelated variable must not affect resolving this token.""" + reason, resolved = checker._resolve_path_tokens(["$DIR"], {"DIR": "sub"}, {}, {"OTHER"}) + assert reason is None + assert resolved == ("sub",) + + def test_classify_extracts_every_historical_path_when_a_checkout_path_is_reassigned_after_use() -> None: """End-to-end regression pin for the round-21 finding at the `classify()` level. Confirmed live before this fix: `F=dirty.py; git @@ -2157,7 +2266,7 @@ def test_rule_array_literal_content_detects_a_braced_subscript_decoy() -> None: the subscript decoy blocked it from ever firing until it collapsed away.""" tokens = ["dummy=", "(", "${NEVERSET[0]}", "uv", "$VERB", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None @@ -2168,7 +2277,7 @@ def test_rule_array_literal_content_detects_a_fused_reference_chain_decoy() -> N before a fused chain of two bare references was recognized as vanishing as a unit.""" tokens = ["dummy=", "(", "$A_UNSET$B_UNSET", "gh", "pr", "merge", "1", ")"] - reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_array_literal_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None @@ -2301,7 +2410,7 @@ def test_rule_command_substitution_content_scans_second_fused_span_in_same_token this test only proves that fix reached end-to-end through `_rule_command_substitution_content`'s own scan loop.""" tokens = ["echo", "$(echo ok)$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None @@ -2310,20 +2419,20 @@ def test_rule_command_substitution_content_skips_blank_fused_span_then_finds_den skipped without denying by itself, but scanning continues to the next fused span in the same token.""" tokens = ["echo", "$( )$(pip install evil-pkg)"] - reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) + reason, _, _ = checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) assert reason is not None def test_rule_command_substitution_content_both_fused_spans_harmless() -> None: tokens = ["echo", "$(echo ok)$(echo also-ok)"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) == (None, False, ()) def test_rule_command_substitution_content_empty_unquoted_span_skipped() -> None: """An empty, unquoted `$()` substitution has no inner tokens to recurse into -- distinct from the fused/quoted empty-span case above.""" tokens = ["$", "(", ")"] - assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}) == (None, False, ()) + assert checker._rule_command_substitution_content(tokens, {}, {}, {}, {}, {}, {}, {}, set()) == (None, False, ()) def test_tokenize_raises_on_unbalanced_quote() -> None: @@ -3047,7 +3156,7 @@ def test_main_allows_a_harmless_command(monkeypatch: pytest.MonkeyPatch, capsys: def test_resolve_path_tokens_returns_literal_tokens_unchanged(paths: list[str]) -> None: """Model-based: every literal (non-dynamic) token is returned as-is, in order, with no deny reason.""" - reason, resolved = checker._resolve_path_tokens(paths, {}, {}) + reason, resolved = checker._resolve_path_tokens(paths, {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3065,7 +3174,7 @@ def test_resolve_path_tokens_resolves_a_braced_reference_case_preserved(name: st against the lowercased map and would have silently mismatched a mixed-case path like `README.md` against `readme.md`.""" mixed_case_value = value.swapcase() - reason, resolved = checker._resolve_path_tokens([f"${{{name}}}"], {name: mixed_case_value}, {}) + reason, resolved = checker._resolve_path_tokens([f"${{{name}}}"], {name: mixed_case_value}, {}, set()) assert reason is None assert resolved == (mixed_case_value,) @@ -3079,7 +3188,7 @@ def test_resolve_path_tokens_denies_an_unresolvable_dynamic_token(name: str) -> `git diff --quiet HEAD -- PATH` exits 0 (clean) for a path that does not exist (issue #1375 Fact 5, confirmed live), which would be fail-open.""" - reason, resolved = checker._resolve_path_tokens([f"${name}"], {}, {}) + reason, resolved = checker._resolve_path_tokens([f"${name}"], {}, {}, set()) assert reason is not None assert resolved == () @@ -3093,7 +3202,7 @@ def test_resolve_path_tokens_denies_an_array_subscript_token() -> None: token's own text UNCHANGED -- silently treating an unexpanded shell construct as though it were already a resolved literal path. Must deny, not pass `${paths[@]}` through as a literal filename.""" - reason, resolved = checker._resolve_path_tokens(["${paths[@]}"], {}, {}) + reason, resolved = checker._resolve_path_tokens(["${paths[@]}"], {}, {}, set()) assert reason is not None assert resolved == () @@ -3103,7 +3212,7 @@ def test_resolve_path_tokens_denies_an_array_subscript_token() -> None: def test_git_checkout_paths_extracts_every_token_after_double_dash(paths: list[str]) -> None: """Model-based, sub-case (a): every token after a literal `--` is a path -- the near-miss's own exact shape (`git checkout -- PATH`).""" - reason, resolved = checker._git_checkout_paths(["--", *paths], {}, {}) + reason, resolved = checker._git_checkout_paths(["--", *paths], {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3113,7 +3222,7 @@ def test_git_checkout_paths_denies_double_dash_with_nothing_following() -> None: harmless no-op in real git by itself, but a downstream pipe/loop could still append paths at runtime this classifier cannot see, and denying a genuine no-op costs nothing.""" - reason, resolved = checker._git_checkout_paths(["--"], {}, {}) + reason, resolved = checker._git_checkout_paths(["--"], {}, {}, set()) assert reason is not None assert resolved == () @@ -3126,7 +3235,7 @@ def test_git_checkout_paths_extracts_two_or_more_positionals_with_no_double_dash `git checkout no-such-ref no-such-file` reports a pathspec error for BOTH arguments, so every position past the first is a pathspec under every resolution real git can take once one exists at all.""" - reason, resolved = checker._git_checkout_paths(paths, {}, {}) + reason, resolved = checker._git_checkout_paths(paths, {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3139,7 +3248,7 @@ def test_git_checkout_paths_treats_a_single_dot_or_dotdot_positional_as_a_path(d (confirmed live: `git check-ref-format --branch .`/`--branch ..` both fail), and `git checkout .` on a dirty tracked file was confirmed live to silently discard the change.""" - reason, resolved = checker._git_checkout_paths([dot], {}, {}) + reason, resolved = checker._git_checkout_paths([dot], {}, {}, set()) assert reason is None assert resolved == (dot,) @@ -3152,7 +3261,7 @@ def test_git_checkout_paths_is_a_non_goal_for_a_single_bare_positional(name: str `git checkout SOMENAME` from a branch/ref name needs a live ref-existence lookup this pure classifier does not perform.""" assume(name not in (".", "..")) - reason, resolved = checker._git_checkout_paths([name], {}, {}) + reason, resolved = checker._git_checkout_paths([name], {}, {}, set()) assert reason is None assert resolved == () @@ -3161,7 +3270,7 @@ def test_git_checkout_paths_allows_a_flag_only_invocation() -> None: """No false positive: `git checkout -b new-branch` has one flag-shaped and one non-flag-shaped token, but the non-flag token is a branch name, not `.`/`..` -- stays the Non-goal, empty paths.""" - reason, resolved = checker._git_checkout_paths(["-b", "new-branch"], {}, {}) + reason, resolved = checker._git_checkout_paths(["-b", "new-branch"], {}, {}, set()) assert reason is None assert resolved == () @@ -3172,7 +3281,7 @@ def test_git_restore_paths_empty_when_staged_without_worktree(staged: str, paths """Model-based: `--staged`/`-S` without `--worktree` never touches the working tree -- empty `checkout_restore_paths`, never live-checked, regardless of what path arguments are also present.""" - reason, resolved = checker._git_restore_paths([staged, *paths], {}, {}) + reason, resolved = checker._git_restore_paths([staged, *paths], {}, {}, set()) assert reason is None assert resolved == () @@ -3184,7 +3293,7 @@ def test_git_restore_paths_checked_when_staged_and_worktree_both_present(worktre --worktree PATH` is a real working-tree-affecting restore despite `--staged` being present -- `saw_worktree=True` must still force the path to be checked.""" - reason, resolved = checker._git_restore_paths(["--staged", worktree, *paths], {}, {}) + reason, resolved = checker._git_restore_paths(["--staged", worktree, *paths], {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3194,7 +3303,7 @@ def test_git_restore_paths_checked_when_staged_and_worktree_both_present(worktre def test_git_restore_paths_checked_with_no_flags_at_all(paths: list[str]) -> None: """Model-based: a bare `git restore PATH` with no flags at all is never staged-only-safe -- always checked.""" - reason, resolved = checker._git_restore_paths(paths, {}, {}) + reason, resolved = checker._git_restore_paths(paths, {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3207,7 +3316,7 @@ def test_git_restore_paths_checked_for_source_short_flag_not_conflated_with_stag (`--staged`, boolean) the way a lower-casing flag scan (like `_is_git_push_segment`'s own) would -- `git restore -s main PATH` stays checked, not wrongly read as staged-only-safe.""" - reason, resolved = checker._git_restore_paths(["-s", ref, *paths], {}, {}) + reason, resolved = checker._git_restore_paths(["-s", ref, *paths], {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3219,7 +3328,7 @@ def test_git_restore_paths_last_occurrence_wins_for_staged(last: str) -> None: --no-staged` ends with `saw_staged=False` (checked), and `--no-staged --staged` ends with `saw_staged=True` (empty, iff no `--worktree`).""" flags = ["--no-staged", "--staged"] if last == "--staged" else ["--staged", "--no-staged"] - reason, resolved = checker._git_restore_paths([*flags, "f.py"], {}, {}) + reason, resolved = checker._git_restore_paths([*flags, "f.py"], {}, {}, set()) assert reason is None if last == "--staged": assert resolved == () @@ -3234,7 +3343,7 @@ def test_git_restore_paths_last_occurrence_wins_for_worktree(paths: list[str]) - `--staged --worktree --no-worktree` ends with `saw_worktree=False`, so the invocation is safe (staged, not worktree) and never live-checked -- exercises the `--no-worktree` branch directly.""" - reason, resolved = checker._git_restore_paths(["--staged", "--worktree", "--no-worktree", *paths], {}, {}) + reason, resolved = checker._git_restore_paths(["--staged", "--worktree", "--no-worktree", *paths], {}, {}, set()) assert reason is None assert resolved == () @@ -3251,7 +3360,7 @@ def test_git_restore_paths_every_boolean_flag_consumes_no_value(flag: str, paths `--ignore-unmerged`, `--ignore-skip-worktree-bits`) is skipped without consuming the token after it as a value -- the following path tokens are still extracted.""" - reason, resolved = checker._git_restore_paths([flag, *paths], {}, {}) + reason, resolved = checker._git_restore_paths([flag, *paths], {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3262,10 +3371,10 @@ def test_git_restore_paths_recurse_submodules_bare_and_fused(value: str, paths: """Model-based: `--recurse-submodules` (bare, consumes nothing) and `--recurse-submodules=VALUE` (fused, self-contained) are both skipped without treating the next token as a value or as part of the flag.""" - reason, resolved = checker._git_restore_paths(["--recurse-submodules", *paths], {}, {}) + reason, resolved = checker._git_restore_paths(["--recurse-submodules", *paths], {}, {}, set()) assert reason is None assert resolved == tuple(paths) - reason, resolved = checker._git_restore_paths([f"--recurse-submodules={value}", *paths], {}, {}) + reason, resolved = checker._git_restore_paths([f"--recurse-submodules={value}", *paths], {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3296,7 +3405,7 @@ def test_git_restore_paths_extracts_every_token_after_double_dash(paths: list[st """`--` disambiguates every remaining token as a pathspec for `git restore`, the identical role it plays for `git checkout` -- must be recognized, not denied as an unrecognized flag.""" - reason, resolved = checker._git_restore_paths(["--", *paths], {}, {}) + reason, resolved = checker._git_restore_paths(["--", *paths], {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3311,7 +3420,7 @@ def test_git_restore_paths_recognizes_fused_value_flags(flag_value: tuple[str, s legitimate git syntax as the separate-token form already recognized -- must not be denied as an unrecognized flag.""" flag, value = flag_value - reason, resolved = checker._git_restore_paths([f"{flag}={value}", *paths], {}, {}) + reason, resolved = checker._git_restore_paths([f"{flag}={value}", *paths], {}, {}, set()) assert reason is None assert resolved == tuple(paths) @@ -3329,7 +3438,7 @@ def test_git_restore_paths_denies_pathspec_from_file(flag: str) -> None: outright rather than silently under-extracting (an empty `checkout_restore_paths` would be exactly issue #1375 Fact 5's own fail-open shape).""" - reason, resolved = checker._git_restore_paths([flag], {}, {}) + reason, resolved = checker._git_restore_paths([flag], {}, {}, set()) assert reason is not None assert resolved == () @@ -3344,7 +3453,7 @@ def test_git_restore_paths_denies_an_unrecognized_flag(flag: str) -> None: assume(flag not in checker._RESTORE_BOOLEAN_FLAGS | checker._RESTORE_VALUE_FLAGS) assume(not flag.startswith("--pathspec-from-file") and not flag.startswith("--recurse-submodules")) assume(flag not in ("--staged", "--no-staged", "--worktree", "--no-worktree")) - reason, resolved = checker._git_restore_paths([flag], {}, {}) + reason, resolved = checker._git_restore_paths([flag], {}, {}, set()) assert reason is not None assert resolved == () @@ -3818,7 +3927,7 @@ def test_git_checkout_paths_folds_branch_creation_flags_into_the_non_goal(flag: discards the change while the old code reported this as checked-safe. Must now fold into the same honest, no-claim Non-goal bare `git checkout SOMENAME` already carries -- empty paths, not a false claim.""" - reason, paths = checker._git_checkout_paths([flag, "newbranch", "other"], {}, {}) + reason, paths = checker._git_checkout_paths([flag, "newbranch", "other"], {}, {}, set()) assert reason is None assert paths == () @@ -3828,7 +3937,7 @@ def test_git_checkout_paths_branch_creation_flag_wins_even_with_a_double_dash() exclusive with every pathspec-checkout mode (per `git checkout -h`'s own synopsis) -- the Non-goal fold must fire before sub-case (a)'s own `--`-present branch is ever reached, not only when `--` is absent.""" - reason, paths = checker._git_checkout_paths(["-b", "newbranch", "--", "file.py"], {}, {}) + reason, paths = checker._git_checkout_paths(["-b", "newbranch", "--", "file.py"], {}, {}, set()) assert reason is None assert paths == () @@ -3837,7 +3946,7 @@ def test_git_checkout_paths_still_extracts_a_real_path_without_a_branch_creation """No regression from the branch-creation fold: an ordinary two- positional pathspec checkout with no `-b`/`-B`/`--orphan` present is unaffected.""" - reason, paths = checker._git_checkout_paths(["a.py", "b.py"], {}, {}) + reason, paths = checker._git_checkout_paths(["a.py", "b.py"], {}, {}, set()) assert reason is None assert paths == ("a.py", "b.py") @@ -3864,7 +3973,7 @@ def test_git_checkout_paths_denies_pathspec_from_file(flag: str) -> None: a file containing the real pathspecs, not an ambiguous ref/path. Live-verified end-to-end that this silently discarded a dirty tracked file listed in the control file.""" - reason, resolved = checker._git_checkout_paths([flag, "files.txt"], {}, {}) + reason, resolved = checker._git_checkout_paths([flag, "files.txt"], {}, {}, set()) assert reason is not None assert resolved == () @@ -3884,7 +3993,7 @@ def test_rule_git_checkout_restore_accumulates_paths_across_segments(command_pat command (`git checkout -- a.py; git restore b.py`) accumulate paths from every segment, not just the first.""" segments = [["git", "checkout", "--", *command_paths], ["git", "restore", *command_paths]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}, set()) assert reason is None assert resolved == (*command_paths, *command_paths) @@ -3897,7 +4006,7 @@ def test_rule_git_checkout_restore_denies_when_git_dir_env_var_assigned() -> Non token-shape fact) rather than letting the live wrapper check the wrong tree.""" segments = [["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"GIT_DIR": "/tmp/x.git"}, {}, {}, {}, set()) assert reason is not None assert resolved == () @@ -3908,7 +4017,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_cd() -> Non the wrapper's own fixed `.cwd` unsound for a LATER checkout/restore segment -- denied outright.""" segments = [["cd", "/tmp"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}, set()) assert reason is not None assert resolved == () @@ -3918,7 +4027,7 @@ def test_rule_git_checkout_restore_allows_cd_after_the_checkout_segment() -> Non `cd` in an EARLIER segment -- a `cd` AFTER the checkout/restore segment does not retroactively make the already-scanned segment unsound.""" segments = [["git", "checkout", "--", "f.py"], ["cd", "/tmp"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}, set()) assert reason is None assert resolved == ("f.py",) @@ -3934,7 +4043,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_is_pushd_or_po claim that the wrapper's live check then found clean at the wrong `.cwd`, silently allowing a real, uncommitted-change discard.""" segments = [[relocator, "/tmp"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}, set()) assert reason is not None assert resolved == () @@ -3959,7 +4068,7 @@ def test_rule_git_checkout_restore_denies_when_an_earlier_segment_starts_with_a_ `checkout_restore_paths` claim the same way round 9's own fix closed for the literal case.""" segments = [["$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}, set()) assert reason is not None assert resolved == () @@ -3971,7 +4080,7 @@ def test_rule_git_checkout_restore_allows_a_genuinely_vanishing_dynamic_word() - real bash would run whatever token follows as the actual command word instead, and that token is scanned on its own merits.""" segments = [["${NEVERSET}", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {}, {}, {}, {}, set()) assert reason is None assert resolved == ("f.py",) @@ -4015,7 +4124,7 @@ def test_rule_git_checkout_restore_allows_a_dynamic_word_resolving_to_something_ not a live production gap, since production always keeps the two dicts' own key sets in sync).""" segments = [["$EDITOR", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}, {}, {"EDITOR": "vim"}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"EDITOR": "vim"}, {}, {"EDITOR": "vim"}, {}, set()) assert reason is None assert resolved == ("f.py",) @@ -4094,7 +4203,7 @@ def test_dynamic_word_may_resolve_to_a_cwd_relocator_true_for_a_still_dynamic_ca def test_rule_git_checkout_restore_denies_a_still_dynamic_candidate() -> None: segments = [["${UNSET:-$OTHER}", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"OTHER": "cd"}, {}, {}, {}, set()) assert reason is not None assert resolved == () @@ -4129,7 +4238,7 @@ def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_v assigned) resolved to a CONFIDENT, WRONG `checkout_restore_paths` claim -- real bash genuinely runs `cd sub` there.""" segments = [["$NEVERSET", "$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}, set()) assert reason is not None assert resolved == () @@ -4172,7 +4281,7 @@ def test_rule_git_checkout_restore_denies_a_dynamic_relocator_behind_a_leading_r Live-verified before this fix: `X=cd; > /dev/null $X sub; git checkout -- dirty.py` resolved to a confident, wrong ALLOW.""" segments = [[">", "/dev/null", "$X", "sub"], ["git", "checkout", "--", "f.py"]] - reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}) + reason, resolved = checker._rule_git_checkout_restore(segments, {"X": "cd"}, {}, {}, {}, set()) assert reason is not None assert resolved == () @@ -4306,13 +4415,13 @@ def test_git_checkout_paths_excludes_a_trailing_redirect_clause() -> None: wrapper check to wrongly deny whenever the unrelated append target happened to be dirty, even though an append redirect can never discard that file's existing content.""" - deny_reason, paths = checker._git_checkout_paths(["--", "f.py", ">>", "unrelated_append_target.py"], {}, {}) + deny_reason, paths = checker._git_checkout_paths(["--", "f.py", ">>", "unrelated_append_target.py"], {}, {}, set()) assert deny_reason is None assert paths == ("f.py",) def test_git_restore_paths_excludes_a_trailing_redirect_clause() -> None: - deny_reason, paths = checker._git_restore_paths(["f.py", ">>", "unrelated_append_target.py"], {}, {}) + deny_reason, paths = checker._git_restore_paths(["f.py", ">>", "unrelated_append_target.py"], {}, {}, set()) assert deny_reason is None assert paths == ("f.py",) @@ -4350,7 +4459,7 @@ def test_git_checkout_paths_does_not_drop_a_digit_shaped_path() -> None: real, dirty, tracked file) -- the classifier's own former digit- consuming redirect heuristic wrongly treated `2` as an fd-redirect prefix rather than a real path argument.""" - deny_reason, paths = checker._git_checkout_paths(["--", "realfile.py", "2", ">", "target.txt"], {}, {}) + deny_reason, paths = checker._git_checkout_paths(["--", "realfile.py", "2", ">", "target.txt"], {}, {}, set()) assert deny_reason is None assert paths == ("realfile.py", "2") @@ -4362,7 +4471,7 @@ def test_git_restore_paths_does_not_drop_a_real_path_behind_a_digit_redirect() - once `2` vanished into the wrongly-recognized redirect, `--source`'s own value-consumption swallowed `file.py` itself, the actual restore target, leaving nothing for the live wrapper check to examine.""" - deny_reason, paths = checker._git_restore_paths(["--source", "2", ">", "target.txt", "file.py"], {}, {}) + deny_reason, paths = checker._git_restore_paths(["--source", "2", ">", "target.txt", "file.py"], {}, {}, set()) assert deny_reason is None assert paths == ("file.py",) @@ -4662,3 +4771,69 @@ def test_classify_leaves_reassigned_but_unrelated_dynamic_word_allowed() -> None invents a match out of nothing.""" verdict = checker.classify("TOOL=echo; $TOOL hello; TOOL=world") assert verdict.deny is False + + +def test_classify_denies_checkout_path_reassigned_to_a_dynamic_value_after_use() -> None: + """End-to-end regression pin for the round-24 finding at the + `classify()` level, top-level shape. Confirmed live before this fix: + `DIR=sub; DIR=$(echo other); git checkout -- $DIR` resolved `checkout_ + restore_paths` to `('sub',)` -- confidently the WRONG, STALE value. + Confirmed live via a real bash proxy (stand-in shell function) that + real bash genuinely resolves `$DIR` to whatever the substitution + evaluates to, not `sub`, at its actual point of use; and end-to-end + through the real wrapper against a scratch repo, wrongly allowed + before this fix, with the uncommitted edit silently discarded once + actually executed.""" + verdict = checker.classify("DIR=sub; DIR=$(echo other); git checkout -- $DIR") + assert verdict.deny is True + + +def test_classify_denies_restore_path_reassigned_to_a_dynamic_value_after_use() -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-24 finding was confirmed live for both subcommands.""" + verdict = checker.classify("DIR=sub; DIR=$(echo other); git restore $DIR") + assert verdict.deny is True + + +def test_classify_denies_a_fused_checkout_path_reassigned_to_a_dynamic_value_after_use() -> None: + """The fused-reference shape (round 23's own scope) combined with the + dynamic-reassignment shape (round 24's own finding): `"$DIR/f"` + resolves through the SAME poisoned name.""" + verdict = checker.classify('DIR=sub; DIR=$(echo other); git checkout -- "$DIR/f"') + assert verdict.deny is True + + +def test_classify_leaves_a_dynamic_reassignment_of_an_unrelated_name_unaffected() -> None: + """No false positive: a dynamic reassignment of a name NOT referenced + by the checkout/restore path argument must not poison an unrelated + resolution.""" + verdict = checker.classify("DIR=sub; OTHER=$(echo x); git checkout -- $DIR") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("sub",) + + +def test_classify_widens_an_indirect_reference_checkout_path_reassigned_after_use() -> None: + """End-to-end regression pin for the round-24 finding's second class, + at the `classify()` level. Confirmed live before this fix: `TARGET= + sub; C=TARGET; git checkout -- ${!C}; TARGET=other` resolved `checkout_ + restore_paths` to `('other',)` alone -- the WRONG, order-blind- + collapsed last value of TARGET (real bash: `${!C}` genuinely was + `sub` at its actual point of use). Confirmed live via a real bash + proxy and end-to-end through the real wrapper against a scratch repo + with `sub` genuinely dirty: wrongly allowed before this fix, with the + uncommitted edit silently discarded once actually executed. `deny` + itself stays False here (this module never unconditionally denies + checkout/restore); the live wrapper's own `git diff --quiet` check + against the now-widened `sub` candidate is what turns this into an + actual deny -- see `hooks/test_gitapex_check_bash_safety.py`'s own + companion end-to-end pin.""" + verdict = checker.classify("TARGET=sub; C=TARGET; git checkout -- ${!C}; TARGET=other") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("sub", "other") + + +def test_classify_widens_an_indirect_reference_restore_path_reassigned_after_use() -> None: + """Companion to the checkout pin above, for `git restore`.""" + verdict = checker.classify("TARGET=sub; C=TARGET; git restore ${!C}; TARGET=other") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("sub", "other") From a9f99780aa77bf66c9f71b13884804fa1a395482 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 20:39:38 +0000 Subject: [PATCH 30/46] fix(hooks): close a compound-append-assignment blind spot in checkout/restore A fresh, independent adversarial review of this PR's current head (round 25) found round 24's own new `_names_with_dynamic_assignment` was itself blind to bash's compound/append-assignment operator (`NAME+=value`): `_ASSIGN_RE` never matches a `+=` token at all (its own greedy identifier-char class cannot consume past the literal `+`, so the required `=` immediately after never lines up), so a name appended to -- with either a dynamic or a static value -- was completely invisible to the reassignment-poisoning check round 24 just added, regardless of whether the appended text was itself dynamic. Independently re-verified live before acting: `DIR=sub; for i in 1; do DIR+=$(echo other); done; git checkout -- $DIR` resolved `checkout_restore_paths` to `('sub',)` -- the STALE, pre-append value. Confirmed via a real bash proxy that `$DIR` genuinely becomes `subother` (`bash -c 'DIR=sub; for i in 1; do DIR+=$(echo other); done; echo $DIR'`), and end-to-end through the real wrapper against a scratch repo with the genuinely-appended-to path dirty: wrongly allowed with exit 0, and actually running the command afterward silently discarded the uncommitted edit. Reproduced identically for `git restore`. Closed by a new `_APPEND_ASSIGN_RE` (matches `NAME+=` by name alone, deliberately unanchored at the end so it still matches the shlex- tokenizer-split form a `NAME+=$(...)` assignment produces), consumed by `_names_with_dynamic_assignment` alongside its existing `_ASSIGN_RE` check. Poisons the name on ANY `+=` occurrence, not only a dynamic one: a static append (`DIR+=txt`) shares the identical exposure for a different reason -- reconstructing the real concatenated value would need genuine execution-order tracking this classifier does not perform (which static assignment among several was actually in effect immediately before the append) -- so it is poisoned the same way, independent of `_is_dynamic`. No new recursive threading needed: this extends the existing, already-fully-threaded `_names_with_dynamic_ assignment` mechanism round 24 built, not a new outer-scope parameter. Regression tests added at every established layer: end-to-end wrapper- level for both checkout and restore against a real scratch repo (hooks/test_gitapex_check_bash_safety.py), and unit/property level for `_names_with_dynamic_assignment`'s dynamic-append, static-append, and unrelated-name cases (including a Hypothesis `@given` property test), plus `classify()`-level end-to-end pins for both subcommands and a false-positive guard (tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI thresholds), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-25 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 58 +++++++++++-- hooks/test_gitapex_check_bash_safety.py | 33 ++++++++ ...st_gitapex_check_bash_safety_properties.py | 81 +++++++++++++++++++ 3 files changed, 166 insertions(+), 6 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 8c36847a..6b3f73fc 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -501,6 +501,18 @@ class (rounds 5-8: shlex's own quote removal; round 9: bash's default- _MULTI_OPS = {"&&", "||"} _ASSIGN_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)=(.*)$") +# Matches bash's own compound/append-assignment operator (`NAME+=value`) +# by NAME alone -- deliberately NOT anchored at the end (`re.match`, not +# `re.fullmatch`), so it still matches the shlex-tokenizer-split form a +# `NAME+=$(...)` assignment produces (e.g. the single token `DIR+=$`, with +# `(`/`echo`/`other`/`)` as their own separate following tokens -- shlex is +# punctuation-aware around `(`/`)`). `_ASSIGN_RE` above never matches a +# `+=` token at all (its own greedy identifier-char class cannot consume +# past the literal `+`, so the required `=` immediately after never lines +# up) -- used only by `_names_with_dynamic_assignment`, see that +# function's own docstring for the live bypass this closes. +_APPEND_ASSIGN_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\+=") + # Matches one `$NAME`/`${NAME}`/`${NAME:-default}`/`${!NAME}` reference # anywhere in a token, capturing its full span (including the braces, when # present) so _substitute_var_refs_candidates below can replace exactly @@ -3756,8 +3768,12 @@ def _multi_valued_names_referenced( def _names_with_dynamic_assignment(tokens: list[str]) -> set[str]: - """Every NAME assigned a DYNAMIC value (containing `$`/backtick) - anywhere in TOKENS. + """Every NAME whose own value, at some point in TOKENS, is genuinely + UNKNOWN to this classifier -- either a DYNAMIC value (containing + `$`/backtick) via a plain `NAME=value` assignment, or ANY compound + `NAME+=value` (append) assignment regardless of whether the appended + text is itself static or dynamic (see the round-25 paragraph below + for why the static case needs the same treatment). CRITICAL bug found by independent adversarial review (round 24, issue #1375) and independently reproduced live: `_assigned_raw_values`/ @@ -3779,15 +3795,45 @@ def _names_with_dynamic_assignment(tokens: list[str]) -> set[str]: uncommitted edit to `other`. Also reproduced identically for the fused-reference form (`"$DIR/f"`). + CRITICAL bug found by independent adversarial review (round 25, issue + #1375) and independently reproduced live: `_ASSIGN_RE` never matches + bash's own `NAME+=value` compound/append-assignment operator at all + (its own greedy identifier-char class cannot consume past the + literal `+`, so the required `=` immediately after never lines up) -- + this function's own round-24 form, keyed entirely off `_ASSIGN_RE`, + was completely blind to an appended name regardless of whether the + appended text was itself dynamic. `DIR=sub; for i in 1; do + DIR+=$(echo other); done; git checkout -- $DIR` resolved + `checkout_restore_paths` to `('sub',)` -- the STALE, pre-append value + (real bash: `$DIR` genuinely becomes `subother`, confirmed live via + `bash -c 'DIR=sub; for i in 1; do DIR+=$(echo other); done; echo + $DIR'`). Confirmed live end-to-end through the real wrapper against a + scratch repo with the genuinely-appended-to path dirty and the stale + pre-append path clean: wrongly allowed with exit 0, and actually + running the command afterward silently discarded the uncommitted + edit. Reproduced identically for `git restore`. A STATIC append + (`DIR+=txt`, no `$`/backtick in the appended text) shares the + identical exposure for a DIFFERENT reason: reconstructing the real + concatenated value would need genuine execution-order tracking this + classifier does not perform (which static assignment among several + was actually in effect immediately before the append), so a static + append is poisoned here too, via `_APPEND_ASSIGN_RE` alone, + independent of `_is_dynamic`. + Closed by `_resolve_path_tokens` treating a name in this set as forced-unresolvable (deny outright) wherever referenced -- the same posture this module already takes for a name that was NEVER assigned - any value at all (a name with a static-only history, or no history, - resolves exactly as before; a name with even one dynamic assignment - anywhere loses all confidence, matching how this classifier already - treats a name with NO recorded static value).""" + any value at all (a name with a static-only history of plain `=` + assignments, or no history, resolves exactly as before; a name with + even one dynamic OR append assignment anywhere loses all confidence, + matching how this classifier already treats a name with NO recorded + static value).""" names: set[str] = set() for token in tokens: + append_match = _APPEND_ASSIGN_RE.match(token) + if append_match: + names.add(append_match.group(1)) + continue if not _is_dynamic(token): continue match = _ASSIGN_RE.match(token) diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 27f66d79..689a1d29 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -2167,6 +2167,39 @@ def test_restore_denied_when_an_indirect_reference_target_is_reassigned_after_us assert result.returncode == 2, f"stderr={result.stderr!r}" +def test_checkout_denied_when_the_path_name_is_dynamically_appended_to_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-25 independent review, issue + #1375). `_ASSIGN_RE` never matches bash's own `NAME+=value` compound/ + append-assignment operator at all, so round 24's own `_names_with_ + dynamic_assignment` -- keyed entirely off `_ASSIGN_RE` -- was + completely blind to an appended name. Live-verified before this fix: + `DIR=dirty.py; for i in 1; do DIR+=$(echo .bak); done; git checkout + -- $DIR` resolved `checkout_restore_paths` to `('dirty.py',)` -- the + STALE, pre-append value, since real bash genuinely resolves `$DIR` to + `dirty.py.bak` at its actual point of use.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py.bak") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run( + "DIR=dirty.py; for i in 1; do DIR+=$(echo .bak); done; git checkout -- $DIR", + payload_cwd=str(repo_dir), + ) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_restore_denied_when_the_path_name_is_dynamically_appended_to_after_use(tmp_path: Path) -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-25 finding was confirmed live for both subcommands.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="dirty.py.bak") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run( + "DIR=dirty.py; for i in 1; do DIR+=$(echo .bak); done; git restore $DIR", + payload_cwd=str(repo_dir), + ) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index a2894b5d..42e651a2 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2124,6 +2124,48 @@ def test_names_with_dynamic_assignment_ignores_a_purely_static_name() -> None: assert checker._names_with_dynamic_assignment(["DIR=sub", "DIR=other"]) == set() +def test_names_with_dynamic_assignment_finds_a_dynamic_compound_append() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-fifth round (issue #1375): `_ASSIGN_RE` never matches + bash's own `NAME+=value` compound/append-assignment operator at all, + so the round-24 form of this function -- keyed entirely off + `_ASSIGN_RE` -- was completely blind to an appended name regardless + of whether the appended text was itself dynamic.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", "DIR+=$other"]) == {"DIR"} + + +def test_names_with_dynamic_assignment_finds_a_static_compound_append() -> None: + """A STATIC append (`DIR+=txt`, no `$`/backtick) shares the identical + exposure for a different reason: reconstructing the real concatenated + value would need genuine execution-order tracking this classifier + does not perform, so a static append is poisoned too, independent of + `_is_dynamic`.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", "DIR+=txt"]) == {"DIR"} + + +def test_names_with_dynamic_assignment_append_ignores_an_unrelated_name() -> None: + """No false positive: an append to one name does not poison a + completely different name.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", "OTHER=x", "OTHER+=y"]) == {"OTHER"} + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, appended_value=_VALUES) +def test_names_with_dynamic_assignment_matches_model_for_a_static_then_appended_reassignment( + name: str, static_value: str, appended_value: str +) -> None: + """Model-based: for ANY identifier assigned a static value and then + appended to (`+=`) with a DYNAMIC value, `_names_with_dynamic_ + assignment` always includes it -- and a second, entirely unrelated + identifier that is only ever assigned statically is never included + alongside it.""" + other_name = name + "_OTHER" + tokens = [f"{name}={static_value}", f"{name}+=$({appended_value})", f"{other_name}={static_value}"] + result = checker._names_with_dynamic_assignment(tokens) + assert name in result + assert other_name not in result + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES) def test_names_with_dynamic_assignment_matches_model_for_a_static_then_dynamic_reassignment( @@ -4837,3 +4879,42 @@ def test_classify_widens_an_indirect_reference_restore_path_reassigned_after_use verdict = checker.classify("TARGET=sub; C=TARGET; git restore ${!C}; TARGET=other") assert verdict.deny is False assert verdict.checkout_restore_paths == ("sub", "other") + + +def test_classify_denies_a_checkout_path_dynamically_appended_to_after_use() -> None: + """End-to-end regression pin for the round-25 finding at the + `classify()` level. Confirmed live before this fix: `DIR=sub; for i + in 1; do DIR+=$(echo other); done; git checkout -- $DIR` resolved + `checkout_restore_paths` to `('sub',)` -- the STALE, pre-append value + (real bash: `$DIR` genuinely becomes `subother`, confirmed live via + `bash -c`). Confirmed live end-to-end through the real wrapper + against a scratch repo with the genuinely-appended-to path dirty: + wrongly allowed before this fix, with the uncommitted edit silently + discarded once actually executed.""" + verdict = checker.classify("DIR=sub; for i in 1; do DIR+=$(echo other); done; git checkout -- $DIR") + assert verdict.deny is True + + +def test_classify_denies_a_restore_path_dynamically_appended_to_after_use() -> None: + """Companion to the checkout pin above, for `git restore` -- the + round-25 finding was confirmed live for both subcommands.""" + verdict = checker.classify("DIR=sub; for i in 1; do DIR+=$(echo other); done; git restore $DIR") + assert verdict.deny is True + + +def test_classify_denies_a_checkout_path_statically_appended_to_after_use() -> None: + """A STATIC append shares the identical exposure (see + `_names_with_dynamic_assignment`'s own docstring): reconstructing the + real concatenated value would need genuine execution-order tracking + this classifier does not perform.""" + verdict = checker.classify("DIR=sub; DIR+=txt; git checkout -- $DIR") + assert verdict.deny is True + + +def test_classify_leaves_an_append_to_an_unrelated_name_unaffected() -> None: + """No false positive: an append to a name NOT referenced by the + checkout/restore path argument must not poison an unrelated + resolution.""" + verdict = checker.classify("DIR=sub; OTHER=x; OTHER+=y; git checkout -- $DIR") + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("sub",) From 50f2b6419bbb1a1844796ef97fabf9dbe2e1b395 Mon Sep 17 00:00:00 2001 From: t Date: Sat, 29 Aug 2026 23:20:24 +0000 Subject: [PATCH 31/46] fix(hooks): close read/readarray/mapfile/printf-v/array-element reassignment blind spots A fresh, independent adversarial review of this PR's current head (round 26) found that this classifier's reassignment-tracking, keyed entirely off `_ASSIGN_RE`/`_APPEND_ASSIGN_RE` (rounds 24-25), is completely blind to every OTHER bash construct that reassigns an existing name: `read NAME`/`readarray NAME`/`mapfile NAME` (reading runtime input into NAME), `printf -v NAME ...` (writing a formatted result into NAME), and `NAME[i]=value`/`NAME[i]+=value` (array-element assignment) -- none of these shapes are matched by `_ASSIGN_RE`/ `_APPEND_ASSIGN_RE` at all, since both anchor immediately after NAME's own bare identifier characters. Independently reproduced live before acting, via `classify()`, real bash execution, and the real wrapper against scratch repos: `DIR=sub; read DIR <<< "other"; git checkout -- $DIR` and `DIR=sub; DIR[0]=other; git checkout -- $DIR` both resolved `checkout_restore_paths` to `('sub',)` -- the stale, pre-reassignment value (real bash: `$DIR` genuinely becomes "other", confirmed via `bash -c 'DIR=sub; read DIR <<< "other"; echo "DIR=[$DIR]"'` -> `DIR=[other]` and `bash -c 'arr=x; arr[0]=other; echo "arr=[$arr]"'` -> `arr=[other]`); wrongly allowed with exit 0 through the real wrapper, silently discarding the uncommitted edit. More severely, because the underlying gap lives in reassignment detection generally, it simultaneously defeated two further HARD-DENY paths: `A=harmless; read A <<< "uv"; B=harmless2; read B <<< "install"; $A $B foo` and `M=GET; M[0]=POST; gh api repos/x/y/issues -X $M` were both wrongly allowed -- confirmed live via a stand-in `uv` binary on PATH (captured argv: "install foo", the genuine pip/uv-install bypass B1a/B1b exist specifically to deny) and via direct `classify()` calls. Closed by extending `_names_with_dynamic_assignment` (consumed by checkout/restore's own `_resolve_path_tokens`, unchanged in behavior) and adding a new, deliberately NARROWER `_names_reassigned_by_ untracked_construct` (array-element assignment plus a new segment-aware `_names_reassigned_by_read_or_printf`, recognizing `read`/`readarray`/ `mapfile`'s target operands and `printf -v NAME`'s separate-token form). The narrower function -- not the wider one -- is what a new `_segment_references_a_name` helper threads into `_rule_gh_api_write` and `_segment_loop_hit` (B1a/B1b), each checked only where their own existing preconditions already apply (a confirmed `gh api` segment; a segment whose own command word is itself dynamic). Threading the WIDER set into those two consumers was tried first and reverted: it regressed an existing, unrelated, deliberately-disclosed bypass (`KNOWN_BYPASS_ COMMANDS`'s own `graphql-mutation-keyword-variable-concatenation` case) into a false-positive deny, since that case's own single dynamic-RHS assignment already satisfied the wider (round-24) class even though gh-api-write's existing `_substitute_var_refs_candidates`-based resolution already handles that class soundly on its own. Simply deleting a poisoned name's dict entry was considered and rejected before writing any code: traced through `_write_method_ candidate_hit`'s own `candidates is None` (fail closed) vs. `candidates == []` (vacuously safe) branches, deletion would make gh-api-write/ B1a/B1b LESS safe, the opposite of this fix's own direction. Regression tests added at every established layer: end-to-end wrapper- level for checkout, restore, gh-api-write, and B1a/B1b against real scratch repos and a stand-in `uv` binary (hooks/test_gitapex_check_ bash_safety.py), plus a false-positive guard confirming an unrelated `read` does not poison an unrelated command; unit, false-positive, and `@given` Hypothesis property tests for `_names_reassigned_by_ untracked_construct`, `_names_reassigned_by_read_or_printf`, and `_segment_references_a_name`, plus `classify()`-level end-to-end pins for every reported shape and the known-bypass regression guard (tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI's own whole-codebase invocation, matching `.github/workflows/test.yml` exactly -- a lone- file xenon run is not representative, its per-file average differs from the codebase-wide one the gate actually enforces), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_ safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-26 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 268 ++++++++++++++- hooks/test_gitapex_check_bash_safety.py | 96 ++++++ ...st_gitapex_check_bash_safety_properties.py | 321 ++++++++++++++++++ 3 files changed, 674 insertions(+), 11 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 6b3f73fc..f120d7cc 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -513,6 +513,25 @@ class (rounds 5-8: shlex's own quote removal; round 9: bash's default- # function's own docstring for the live bypass this closes. _APPEND_ASSIGN_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\+=") +# Matches bash's own array-element assignment (`NAME[subscript]=value` or +# `NAME[subscript]+=value`) by NAME alone -- neither `_ASSIGN_RE` nor +# `_APPEND_ASSIGN_RE` matches this shape at all (both anchor immediately +# after NAME's own identifier characters, with no `[...]` in between) -- +# used only by `_names_with_dynamic_assignment`, see that function's own +# docstring (round 26) for the live bypass this closes. +_ARRAY_ELEMENT_ASSIGN_RE = re.compile(r"^([A-Za-z_][A-Za-z0-9_]*)\[[^\]]*\]\+?=") + +# Command words whose whole purpose is to reassign one or more existing +# names from runtime input (`read`/`readarray`/`mapfile`'s own target +# operands) -- see `_names_with_dynamic_assignment`'s own docstring +# (round 26) for how these are scanned and why. +_READ_COMMAND_WORDS = frozenset({"read", "readarray", "mapfile"}) + +# A bare, unqualified bash identifier -- used only to recognize a `read`/ +# `readarray`/`mapfile`/`printf -v` operand as a plausible reassignment +# target, not to validate real bash identifier rules exhaustively. +_BARE_IDENTIFIER_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_]*$") + # Matches one `$NAME`/`${NAME}`/`${NAME:-default}`/`${!NAME}` reference # anywhere in a token, capturing its full span (including the braces, when # present) so _substitute_var_refs_candidates below can replace exactly @@ -3449,8 +3468,50 @@ def _gh_api_field_fused_flagname_dynamic_hit( return False +def _segment_references_a_name(seg: list[str], names: set[str]) -> bool: + """True when any token in SEG references (as a bare `$NAME`, braced + `${NAME}`, default-clause `${NAME:-x}`, or indirect `${!NAME}` + reference) any name in NAMES. A coarse, single-level scan via the + same `_VAR_REF_FULL_RE` `_referenced_names` uses for its own first + level -- deliberately NOT that function's full two-level indirect + expansion (this only needs to know whether a POISONED name itself is + mentioned, not what an indirect reference through it might resolve + to), so it needs neither NAME_TO_RAW_VALUE nor NAME_TO_RAW_VALUE_ + HISTORY -- both absent from `_rule_gh_api_write`'s and `_segment_ + loop_hit`'s own signatures. + + Added by round 26 (issue #1375) alongside `_names_with_dynamic_ + assignment`'s own extension -- see that function's own docstring for + the live `_rule_gh_api_write`/B1a/B1b bypasses this closes, and for + why simply deleting a poisoned name's dict entry (the first approach + considered) would have made those two consumers LESS safe rather + than more.""" + if not names: + return False + for token in seg: + for match in _VAR_REF_FULL_RE.finditer(token): + braced_name, default_name, _default_text, indirect_name, unbraced_run = match.groups() + for name in (braced_name, default_name, indirect_name): + if name is not None and name in names: + return True + if unbraced_run is not None and any(unbraced_run[:i] in names for i in range(len(unbraced_run), 0, -1)): + return True + return False + + +_POISONED_REASSIGNMENT_GH_API_HIT = ( + "a 'gh api' call references a variable whose value was reassigned via a construct this classifier " + "cannot track (`read`/`readarray`/`mapfile`, `printf -v`, or an array-element assignment) -- rewrite " + "as a plain literal command so it can be checked" +) + + def _rule_gh_api_write( - segments: list[list[str]], lowered_command: str, name_to_value: dict[str, str], name_to_raw_value: dict[str, str] + segments: list[list[str]], + lowered_command: str, + name_to_value: dict[str, str], + name_to_raw_value: dict[str, str], + names_reassigned_by_untracked_construct: set[str] | None = None, ) -> str | None: """`literals` is already lowercased, matching the predecessor script's own case-insensitive match against its whole lowered command -- so @@ -3460,12 +3521,26 @@ def _rule_gh_api_write( --raw-field field flag -- literal, dynamic-value, dynamic-flag-name, and fused-flag-name-and-value, per side); kept deliberately thin (each pass owns its own branching) so this function's own cyclomatic - complexity stays low.""" + complexity stays low. + + NAMES_REASSIGNED_BY_UNTRACKED_CONSTRUCT (round 26, issue #1375) + defaults to `None` (treated as empty) so every pre-existing call site + keeps its exact prior behavior; `_classify_tokens`'s own call sites + are the only ones that supply it, deliberately NOT the same (wider) + NAMES_WITH_DYNAMIC_ASSIGNMENT `_resolve_path_tokens` (checkout/ + restore) consumes -- see `_names_reassigned_by_untracked_construct`'s + own docstring for why. See `_segment_references_a_name`'s own + docstring for why a poisoned name is checked here, once has_gh_api is + already confirmed, rather than by deleting the name from NAME_TO_ + VALUE/NAME_TO_RAW_VALUE.""" + poisoned = names_reassigned_by_untracked_construct or set() for seg in segments: literals = [t.lower() for t in seg if not _is_dynamic(t)] has_gh_api = any(literals[i : i + 2] == ["gh", "api"] for i in range(len(literals) - 1)) if not has_gh_api: continue + if _segment_references_a_name(seg, poisoned): + return _POISONED_REASSIGNMENT_GH_API_HIT has_graphql = any(literals[i : i + 3] == ["gh", "api", "graphql"] for i in range(len(literals) - 2)) if has_graphql and "mutation" in lowered_command: return "a 'gh api graphql' call containing a 'mutation' keyword" @@ -3827,7 +3902,63 @@ def _names_with_dynamic_assignment(tokens: list[str]) -> set[str]: assignments, or no history, resolves exactly as before; a name with even one dynamic OR append assignment anywhere loses all confidence, matching how this classifier already treats a name with NO recorded - static value).""" + static value). + + CRITICAL bug found by independent adversarial review (round 26, issue + #1375) and independently reproduced live, both via `classify()` and + via real bash execution: this function's own round-24/25 form, keyed + entirely off `_ASSIGN_RE`/`_APPEND_ASSIGN_RE`, is completely BLIND to + every OTHER bash construct that reassigns an existing name -- `read + NAME`/`readarray NAME`/`mapfile NAME` (reading runtime input into + NAME), `printf -v NAME ...` (writing a formatted result into NAME), + and `NAME[i]=value`/`NAME[i]+=value` (array-element assignment) + neither of `_ASSIGN_RE`/`_APPEND_ASSIGN_RE` matches at all (both + anchor immediately after NAME's own bare identifier characters, with + no `read `/`-v `/`[i]` in between). `DIR=sub; read DIR <<< "other"; + git checkout -- $DIR` and `DIR=sub; DIR[0]=other; git checkout -- + $DIR` both resolved `checkout_restore_paths` to `('sub',)` -- the + STALE, pre-reassignment value (real bash: `$DIR` genuinely becomes + "other" in both cases, confirmed live via `bash -c 'DIR=sub; read DIR + <<< "other"; echo "DIR=[$DIR]"'` -> `DIR=[other]` and `bash -c + 'arr=x; arr[0]=other; echo "arr=[$arr]"'` -> `arr=[other]`). + Independently confirmed live end-to-end through the real wrapper + against a scratch repo, same wrongly-allowed-then-silently-discarded + shape as every prior round in this family. + + More severely, the SAME shapes (`read`/`readarray`/`mapfile`/ + `printf -v`/array-element assignment) ALSO defeat `_rule_gh_api_write` + and `_segment_loop_hit` (B1a/B1b): `A=harmless; read A <<< "uv"; + B=harmless2; read B <<< "install"; $A $B foo` and `M=GET; M[0]=POST; + gh api repos/x/y/issues -X $M` were both wrongly ALLOWED -- confirmed + live via a stand-in `uv` binary on PATH (captured argv: `uv called + with: install foo`, the genuine `pip`/`uv install` bypass B1a/B1b + exist specifically to deny) and via direct `classify()` calls + (`reason == 'no denied pattern matched'` for the `gh api` case). + `_names_reassigned_by_untracked_construct` (see its own docstring) is + the DEDICATED, NARROWER function those two consumers actually use -- + NOT this function's own full union. Threading the FULL union used + here (which also includes the round-24/25 dynamic-RHS-assignment/ + append classes) into gh-api-write/B1a-B1b was tried first and + reverted: `A=muta; B=tion; Q="${A}${B} { x }"; gh api graphql -f + query="$Q"` -- an EXISTING, deliberately-disclosed, unrelated bypass + (`KNOWN_BYPASS_COMMANDS`'s own `graphql-mutation-keyword-variable- + concatenation` case, predating round 26 entirely) -- regressed to a + wrongly-denied false positive, since `Q`'s own single dynamic + assignment already satisfied the round-24 class (ANY dynamic-RHS `=` + assignment, not specifically a REASSIGNMENT after a static one) even + though gh-api-write's own EXISTING `_substitute_var_refs_candidates`- + based resolution already handles that class soundly (an unresolvable + candidate list reads as "no evidence of a write," not "deny + outright" -- the deliberately different, narrower posture this rule + already had for that specific, previously-accepted gap). `_segment_ + references_a_name` (see its own docstring) is the mechanism `_rule_ + gh_api_write`/`_segment_loop_hit` use to apply the NARROWER poisoned- + names set to their own scope, since simply deleting a poisoned name's + dict entry -- traced through `_write_method_candidate_hit`'s own + `candidates is None: True` vs. `candidates == []: False` branches + before writing any code -- would make those two consumers LESS safe + (an empty candidate list reads as "vacuously safe," the opposite of + the fail-closed direction this fix needs).""" names: set[str] = set() for token in tokens: append_match = _APPEND_ASSIGN_RE.match(token) @@ -3839,6 +3970,74 @@ def _names_with_dynamic_assignment(tokens: list[str]) -> set[str]: match = _ASSIGN_RE.match(token) if match: names.add(match.group(1)) + names.update(_names_reassigned_by_untracked_construct(tokens)) + return names + + +def _names_reassigned_by_untracked_construct(tokens: list[str]) -> set[str]: + """Every NAME reassigned in TOKENS via a bash construct this + classifier's `_ASSIGN_RE`/`_APPEND_ASSIGN_RE` (the round-24/25 classes + `_names_with_dynamic_assignment` also covers) do not recognize AT + ALL: array-element assignment (`NAME[i]=value`/`NAME[i]+=value`, via + `_ARRAY_ELEMENT_ASSIGN_RE`) and `read`/`readarray`/`mapfile`/ + `printf -v` reassignment (via `_names_reassigned_by_read_or_printf`). + + Deliberately NARROWER than `_names_with_dynamic_assignment`'s own + full union -- this is the set `_rule_gh_api_write`/`_segment_loop_ + hit` (B1a/B1b) actually consume, NOT the full one `_resolve_path_ + tokens` (checkout/restore) consumes -- see `_names_with_dynamic_ + assignment`'s own round-26 paragraph for why threading the FULL union + into those two HARD-DENY consumers regressed an existing, unrelated, + deliberately-disclosed bypass. Computed fresh from TOKENS only (no + outer-scope union, unlike `_names_with_dynamic_assignment`'s own + recursive threading) -- a `read`/array-element reassignment occurring + OUTSIDE a `$(...)`/array-literal span but referenced only INSIDE it + is a disclosed, narrower residual this pass does not cover; every + round-26 finding actually verified live was a flat, top-level case, + which this fully covers.""" + names: set[str] = set() + for token in tokens: + array_match = _ARRAY_ELEMENT_ASSIGN_RE.match(token) + if array_match: + names.add(array_match.group(1)) + names.update(_names_reassigned_by_read_or_printf(tokens)) + return names + + +def _names_reassigned_by_read_or_printf(tokens: list[str]) -> set[str]: + """Every NAME that `read`/`readarray`/`mapfile` (any bare-identifier- + shaped operand following the command word, skipping option flags, + which never match `_BARE_IDENTIFIER_RE` since they start with `-`) or + `printf -v NAME` (the separate-token `-v` form only -- the fused + `-vNAME` form is a known, disclosed residual, not attempted here) can + reassign somewhere in TOKENS. Segment-aware (via `segment_tokens`) so + a `read`/`printf` command word is only recognized as one when it + genuinely starts its own simple command, not merely because the + literal word appears as an ordinary argument elsewhere. + + Deliberately coarse and over-inclusive rather than modeling each + command's own exact flag grammar (e.g. `read -d END VAR` also treats + "END" as a candidate name, since this function does not know `-d` + takes its own value argument): see `_names_with_dynamic_assignment`'s + own docstring for why a name added here only ever makes this + classifier MORE conservative wherever it is later referenced, never + less -- over-including a flag's own literal argument costs nothing + but a redundant, harmless entry.""" + names: set[str] = set() + for seg in segment_tokens(tokens): + if not seg or _is_dynamic(seg[0]): + continue + head = seg[0].lower() + if head in _READ_COMMAND_WORDS: + for tok in seg[1:]: + if not _is_dynamic(tok) and _BARE_IDENTIFIER_RE.match(tok): + names.add(tok) + elif head == "printf": + for i, tok in enumerate(seg): + if tok == "-v" and i + 1 < len(seg): + target = seg[i + 1] + if not _is_dynamic(target) and _BARE_IDENTIFIER_RE.match(target): + names.add(target) return names @@ -5084,7 +5283,10 @@ def _rule_b2_watched_tool_dynamic_verb_position(seg: list[str]) -> bool: def _segment_loop_hit( - segments: list[list[str]], name_to_value: dict[str, str], name_to_raw_value: dict[str, str] + segments: list[list[str]], + name_to_value: dict[str, str], + name_to_raw_value: dict[str, str], + names_reassigned_by_untracked_construct: set[str] | None = None, ) -> tuple[str | None, bool]: """The B1a/B1b/B2/obfuscated-git-push-second-token loop -- factored out of `_classify_tokens` so it can be run TWICE: once against @@ -5109,9 +5311,34 @@ def _segment_loop_hit( rule at all (confirmed live: even a plain `curl | bash` with no decoy already classifies "no denied pattern matched" here), so there is no equivalent gap for that shape in this file specifically -- only - B2's own literal-`seg[0]` requirement is affected.""" + B2's own literal-`seg[0]` requirement is affected. + + NAMES_REASSIGNED_BY_UNTRACKED_CONSTRUCT (round 26, issue #1375) + defaults to `None` (treated as empty) so every pre-existing call site + keeps its exact prior behavior; `_classify_tokens`'s own call sites + are the only ones that supply it, deliberately NOT the same (wider) + NAMES_WITH_DYNAMIC_ASSIGNMENT `_resolve_path_tokens` (checkout/ + restore) consumes -- see `_names_reassigned_by_untracked_construct`'s + own docstring for why. Checked ONLY when `seg[0]` is itself dynamic + -- the same precondition B1a/B1b already require before either even + runs (see each rule's own docstring) -- so a poisoned name referenced + in an otherwise-harmless, literal-command-word segment (e.g. `echo + $M` where M was reassigned via `read`) is never flagged here: B1a/ + B1b's own bypass only exists where a poisoned name feeds the SAME + position (the dynamically-constructed command word itself, or a + same-segment verb token) those two rules already resolve, so this + check is scoped identically rather than treating every reference + anywhere as a hit.""" is_git_push = False + poisoned = names_reassigned_by_untracked_construct or set() for seg in segments: + if seg and _is_dynamic(seg[0]) and _segment_references_a_name(seg, poisoned): + return ( + "a Bash command word is dynamically constructed from a variable whose value was " + "reassigned via a construct this classifier cannot track (`read`, `printf -v`, or an " + "array-element assignment) -- rewrite as a plain literal command so it can be checked", + is_git_push, + ) if _rule_b1a_dynamic_word_same_segment_verb(seg, _WATCHED_VERBS, name_to_value, name_to_raw_value): return ( "a Bash command word is dynamically constructed, alongside a denied verb literally " @@ -5371,6 +5598,12 @@ def _classify_tokens( **_assigned_raw_values_biased_toward(tokens, _WATCHED_WRITE_BIAS), } names_with_dynamic_assignment = outer_dynamic_names | _names_with_dynamic_assignment(tokens) + # Deliberately narrower than NAMES_WITH_DYNAMIC_ASSIGNMENT above, and + # computed fresh from TOKENS only (no outer-scope union) -- see + # `_names_reassigned_by_untracked_construct`'s own docstring for why + # `_rule_gh_api_write`/`_segment_loop_hit` below consume THIS set, + # not the full one just above. + names_reassigned_by_untracked_construct = _names_reassigned_by_untracked_construct(tokens) lowered_command = " ".join(tokens).lower() is_git_push = is_git_push or any(_is_git_push_segment(seg, raw_assigned) for seg in segments) @@ -5379,17 +5612,25 @@ def _classify_tokens( if literal_hit: return Verdict(True, literal_hit, is_git_push, checkout_restore_paths) - gh_api_hit = _rule_gh_api_write(segments, lowered_command, assigned, raw_assigned) or _rule_gh_api_write( - segments, lowered_command, assigned_write_biased, raw_assigned_write_biased + gh_api_hit = _rule_gh_api_write( + segments, lowered_command, assigned, raw_assigned, names_reassigned_by_untracked_construct + ) or _rule_gh_api_write( + segments, + lowered_command, + assigned_write_biased, + raw_assigned_write_biased, + names_reassigned_by_untracked_construct, ) if gh_api_hit: return Verdict(True, gh_api_hit, is_git_push, checkout_restore_paths) - loop_hit, loop_is_git_push = _segment_loop_hit(segments, assigned, raw_assigned) + loop_hit, loop_is_git_push = _segment_loop_hit( + segments, assigned, raw_assigned, names_reassigned_by_untracked_construct + ) is_git_push = is_git_push or loop_is_git_push if not loop_hit: loop_hit, biased_loop_is_git_push = _segment_loop_hit( - segments, assigned_write_biased, raw_assigned_write_biased + segments, assigned_write_biased, raw_assigned_write_biased, names_reassigned_by_untracked_construct ) is_git_push = is_git_push or biased_loop_is_git_push if loop_hit: @@ -5399,11 +5640,16 @@ def _classify_tokens( collapsed for seg in segments if (collapsed := _strip_leading_unassigned_bare_refs(seg, raw_assigned)) ] if collapsed_segments != segments: - collapsed_hit, collapsed_is_git_push = _segment_loop_hit(collapsed_segments, assigned, raw_assigned) + collapsed_hit, collapsed_is_git_push = _segment_loop_hit( + collapsed_segments, assigned, raw_assigned, names_reassigned_by_untracked_construct + ) is_git_push = is_git_push or collapsed_is_git_push if not collapsed_hit: collapsed_hit, collapsed_biased_is_git_push = _segment_loop_hit( - collapsed_segments, assigned_write_biased, raw_assigned_write_biased + collapsed_segments, + assigned_write_biased, + raw_assigned_write_biased, + names_reassigned_by_untracked_construct, ) is_git_push = is_git_push or collapsed_biased_is_git_push if collapsed_hit: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 689a1d29..d0a081dc 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -230,6 +230,12 @@ def assert_allowed(command: str) -> None: # (issue #1326, ninth round): a default-clause value resolving to a # read method must stay allowed. ("gh api repos/x/y -X${UNSET_VAR-GET}", "gh-api-method-value-default-clause-read"), + # False-positive guard for the round-26 `read`/array-element/ + # `printf -v` reassignment-poisoning fix (issue #1375): `read`-ing (or + # array-assigning, or `printf -v`-ing) into a name never referenced by + # this call at all -- a totally unrelated name -- must not poison this + # unrelated `gh api` read call. + ('read UNRELATED <<< "x"; gh api repos/o/r/issues', "gh-api-get-unrelated-name-read-into"), ] ALLOWED_ORDINARY_COMMANDS = [ @@ -315,6 +321,12 @@ def assert_allowed(command: str) -> None: # `IFS=x; REAL=foo; $REAL uv $VERB` real-expands to `foo uv`, never # touching the watched `uv` tool in dynamic-verb position. ("IFS=x; REAL=foo; $REAL uv $VERB", "dynamic-wrapper-stays-allowed-despite-unrelated-ifs-reassignment"), + # False-positive guard for the round-26 `read`/array-element/ + # `printf -v` reassignment-poisoning fix (issue #1375): the new + # poisoning check in `_segment_loop_hit` is scoped to segments whose + # OWN command word (`seg[0]`) is itself dynamic -- an unrelated `read` + # elsewhere in the command must stay allowed. + ('read UNRELATED <<< "x"; echo hello', "read-into-unrelated-name-stays-allowed"), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -892,6 +904,41 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "A=uv; x=$($A install foo); A=somethingelse", "var-split-tool-and-verb-reassigned-after-use-across-command-substitution", ), + # Found live by Step 8 independent review, twenty-sixth round (issue + # #1375): neither `_ASSIGN_RE` nor `_APPEND_ASSIGN_RE` recognizes + # bash's own `read NAME` builtin or `NAME[i]=value` array-element + # assignment as a reassignment at all -- round 22's own fix (the two + # cases immediately above) only closed the reassignment-ambiguity gap + # for RECOGNIZED assignment tokens (plain `=`/`+=`), leaving both of + # these completely unrecognized shapes open. Confirmed live via a + # stand-in `uv` binary on PATH (captured argv: "install foo") that + # `A=harmless; read A <<< "uv"; B=harmless2; read B <<< "install"; $A + # $B foo` genuinely runs `uv install foo`. + ( + 'A=harmless; read A <<< "uv"; B=harmless2; read B <<< "install"; $A $B foo', + "var-split-tool-and-verb-reassigned-via-read", + ), + # Same round, the array-element-assignment counterpart. + ( + "A=harmless; A[0]=uv; B=harmless2; B[0]=install; $A $B foo", + "var-split-tool-and-verb-reassigned-via-array-element", + ), + # Same round, the gh-api-write counterpart for `read`: `$M` genuinely + # was "POST" at its actual point of use; real bash genuinely ran `gh + # api repos/o/r/pulls/1/merge -X POST`. + ( + 'M=GET; read M <<< "POST"; gh api repos/o/r/pulls/1/merge -X $M', + "gh-api-method-value-reassigned-via-read", + ), + # Same round, the gh-api-write counterpart for array-element + # assignment. + ("M=GET; M[0]=POST; gh api repos/o/r/pulls/1/merge -X $M", "gh-api-method-value-reassigned-via-array-element"), + # Same round, the `printf -v` counterpart (writes a formatted result + # into NAME, equally invisible to `_ASSIGN_RE`/`_APPEND_ASSIGN_RE`). + ( + 'M=GET; printf -v M "%s" POST; gh api repos/o/r/pulls/1/merge -X $M', + "gh-api-method-value-reassigned-via-printf-v", + ), ] @@ -2200,6 +2247,55 @@ def test_restore_denied_when_the_path_name_is_dynamically_appended_to_after_use( assert result.returncode == 2, f"stderr={result.stderr!r}" +def test_checkout_denied_when_the_path_name_is_reassigned_via_read_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-26 independent review, issue + #1375). Neither `_ASSIGN_RE` nor `_APPEND_ASSIGN_RE` recognizes bash's + own `read NAME` builtin as a reassignment at all, so round 25's own + `_names_with_dynamic_assignment` was completely blind to a name + reassigned this way. Live-verified before this fix: `DIR=sub; read + DIR <<< "other"; git checkout -- $DIR` resolved `checkout_restore_ + paths` to `('sub',)` -- the STALE, pre-reassignment value, since real + bash genuinely resolves `$DIR` to `other` at its actual point of use + (confirmed via `bash -c 'DIR=sub; read DIR <<< "other"; echo + "DIR=[$DIR]"'` -> `DIR=[other]`).""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="other") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run( + 'DIR=sub; read DIR <<< "other"; git checkout -- $DIR', + payload_cwd=str(repo_dir), + ) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_restore_denied_when_the_path_name_is_reassigned_via_read_after_use(tmp_path: Path) -> None: + """Companion to the checkout pin above, for `git restore`.""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="other") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run( + 'DIR=sub; read DIR <<< "other"; git restore $DIR', + payload_cwd=str(repo_dir), + ) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + +def test_checkout_denied_when_the_path_name_is_reassigned_via_an_array_element_after_use(tmp_path: Path) -> None: + """CRITICAL bypass regression pin (round-26 independent review, issue + #1375). Bash's own array-element assignment (`NAME[i]=value`) is + invisible to `_ASSIGN_RE`/`_APPEND_ASSIGN_RE` -- both anchor + immediately after NAME's own identifier characters, with no `[...]` + in between. Live-verified before this fix: `arr=x; arr[0]=other; git + checkout -- $arr` resolved `checkout_restore_paths` to `('x',)` -- + the STALE, pre-reassignment value (confirmed via `bash -c 'arr=x; + arr[0]=other; echo "arr=[$arr]"'` -> `arr=[other]`).""" + repo_dir = tmp_path / "repo" + file_path = _init_repo_with_committed_file(repo_dir, filename="other") + file_path.write_text("UNCOMMITTED WORK -- must not be discarded\n") + result = run("arr=x; arr[0]=other; git checkout -- $arr", payload_cwd=str(repo_dir)) + assert result.returncode == 2, f"stderr={result.stderr!r}" + + def test_checkout_denied_in_a_real_merge_conflict_names_the_conflict_remedy(tmp_path: Path) -> None: """A real merge conflict (issue #1375's own Acceptance Criteria Map): the deny message names a remedy that actually works mid-conflict diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 42e651a2..56b907c0 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2182,6 +2182,327 @@ def test_names_with_dynamic_assignment_matches_model_for_a_static_then_dynamic_r assert other_name not in result +def test_names_with_dynamic_assignment_finds_an_array_element_reassignment() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-sixth round (issue #1375): neither `_ASSIGN_RE` nor + `_APPEND_ASSIGN_RE` matches bash's own array-element assignment + (`NAME[i]=value`/`NAME[i]+=value`) at all -- both anchor immediately + after NAME's own bare identifier characters, with no `[...]` in + between. Confirmed live: `bash -c 'arr=x; arr[0]=other; echo + "arr=[$arr]"'` -> `arr=[other]`.""" + assert checker._names_with_dynamic_assignment(["arr=x", "arr[0]=other"]) == {"arr"} + + +def test_names_with_dynamic_assignment_finds_an_array_element_append() -> None: + """Companion to the plain array-element-assignment pin above, for the + `+=` compound form (`NAME[i]+=value`).""" + assert checker._names_with_dynamic_assignment(["arr=x", "arr[0]+=other"]) == {"arr"} + + +def test_names_with_dynamic_assignment_array_element_ignores_an_unrelated_name() -> None: + """No false positive: an array-element assignment to one name does not + poison a completely different name.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", "arr[0]=other"]) == {"arr"} + + +def test_names_with_dynamic_assignment_finds_a_read_reassignment() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-sixth round (issue #1375): `read NAME` reassigns NAME + from runtime input, invisible to `_ASSIGN_RE`/`_APPEND_ASSIGN_RE` + entirely. Confirmed live: `bash -c 'DIR=sub; read DIR <<< "other"; + echo "DIR=[$DIR]"'` -> `DIR=[other]`.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", ";", "read", "DIR"]) == {"DIR"} + + +def test_names_with_dynamic_assignment_finds_a_readarray_reassignment() -> None: + """Companion to the `read` pin above, for `readarray`/`mapfile`'s own + equivalent target-operand shape.""" + assert checker._names_with_dynamic_assignment(["ARR=x", ";", "readarray", "ARR"]) == {"ARR"} + assert checker._names_with_dynamic_assignment(["ARR=x", ";", "mapfile", "ARR"]) == {"ARR"} + + +def test_names_with_dynamic_assignment_read_ignores_flag_tokens() -> None: + """No false positive from the flag tokens themselves: `-r`/`-a` never + match `_BARE_IDENTIFIER_RE` (both start with `-`), so only the real + target name is poisoned.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", ";", "read", "-r", "DIR"]) == {"DIR"} + + +def test_names_with_dynamic_assignment_read_command_word_must_start_its_own_segment() -> None: + """No false positive: the literal word `read` appearing as an ordinary + argument (not as a command word starting its own segment) must not be + treated as the `read` builtin.""" + assert checker._names_with_dynamic_assignment(["echo", "read", "DIR"]) == set() + + +def test_names_with_dynamic_assignment_finds_a_printf_v_reassignment() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-sixth round (issue #1375): `printf -v NAME` writes a + formatted result into NAME, invisible to `_ASSIGN_RE`/`_APPEND_ + ASSIGN_RE` entirely. Confirmed live: `bash -c 'DIR=sub; printf -v DIR + "%s" other; echo "DIR=[$DIR]"'` -> `DIR=[other]`.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", ";", "printf", "-v", "DIR", "%s", "other"]) == {"DIR"} + + +def test_names_with_dynamic_assignment_printf_without_v_flag_is_unaffected() -> None: + """No false positive: an ordinary `printf` call with no `-v` flag at + all must not poison any name.""" + assert checker._names_with_dynamic_assignment(["DIR=sub", ";", "printf", "%s\\n", "DIR"]) == set() + + +def test_names_reassigned_by_read_or_printf_ignores_a_dynamic_printf_v_target() -> None: + """No false positive, and an inherent limit disclosed rather than + silently mishandled: `printf -v $UNKNOWN ...` names its OWN + reassignment target dynamically -- this function cannot know which + real name that resolves to, so it poisons nothing (the same posture + `read $UNKNOWN` would need, though this module does not attempt that + shape either). Confirmed via `tokenize` that `-v`'s own following + token here is genuinely dynamic, not a bare identifier.""" + assert checker._names_reassigned_by_read_or_printf(checker.tokenize('printf -v $X "%s" y')) == set() + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, other_value=_VALUES) +def test_names_with_dynamic_assignment_matches_model_for_a_read_reassignment( + name: str, static_value: str, other_value: str +) -> None: + """Model-based: for ANY identifier assigned a static value and then + reassigned via `read`, `_names_with_dynamic_assignment` always + includes it -- and a second, entirely unrelated identifier that is + only ever assigned statically is never included alongside it.""" + other_name = name + "_OTHER" + tokens = [f"{name}={static_value}", ";", "read", name, ";", f"{other_name}={other_value}"] + result = checker._names_with_dynamic_assignment(tokens) + assert name in result + assert other_name not in result + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, other_value=_VALUES) +def test_names_reassigned_by_untracked_construct_matches_model_for_a_read_reassignment( + name: str, static_value: str, other_value: str +) -> None: + """Model-based, exercising the DEDICATED (narrower) round-26 function + directly rather than only through `_names_with_dynamic_assignment`'s + own wider union: for ANY identifier assigned a static value and then + reassigned via `read`, `_names_reassigned_by_untracked_construct` + always includes it -- and a second, entirely unrelated identifier + that is only ever assigned statically is never included alongside + it.""" + other_name = name + "_OTHER" + tokens = [f"{name}={static_value}", ";", "read", name, ";", f"{other_name}={other_value}"] + result = checker._names_reassigned_by_untracked_construct(tokens) + assert name in result + assert other_name not in result + + +@_PROPERTIES +@given(name=_IDENTIFIERS, subscript=st.sampled_from(["0", "1", "@", "i"]), value=_VALUES) +def test_names_reassigned_by_untracked_construct_matches_model_for_an_array_element_assignment( + name: str, subscript: str, value: str +) -> None: + """Model-based: for ANY identifier assigned an array-element value + (`NAME[subscript]=value`), `_names_reassigned_by_untracked_construct` + always includes NAME -- and a second, entirely unrelated identifier + that is never array-assigned is never included alongside it.""" + other_name = name + "_OTHER" + tokens = [f"{name}[{subscript}]={value}", f"{other_name}={value}"] + result = checker._names_reassigned_by_untracked_construct(tokens) + assert name in result + assert other_name not in result + + +@_PROPERTIES +@given(name=_IDENTIFIERS, target_value=_VALUES) +def test_names_reassigned_by_read_or_printf_matches_model_for_a_read_target(name: str, target_value: str) -> None: + """Model-based, exercising `_names_reassigned_by_read_or_printf` + directly: for ANY identifier that `read` targets, it is always + included -- and a second, entirely unrelated identifier only ever + assigned statically is never included alongside it.""" + other_name = name + "_OTHER" + tokens = ["read", name, ";", f"{other_name}={target_value}"] + result = checker._names_reassigned_by_read_or_printf(tokens) + assert name in result + assert other_name not in result + + +@_PROPERTIES +@given(name=_IDENTIFIERS, format_value=_VALUES) +def test_names_reassigned_by_read_or_printf_matches_model_for_a_printf_v_target(name: str, format_value: str) -> None: + """Model-based: for ANY identifier `printf -v` targets, it is always + included -- and a second, entirely unrelated identifier is never + included alongside it.""" + other_name = name + "_OTHER" + tokens = ["printf", "-v", name, "%s", format_value, ";", f"{other_name}={format_value}"] + result = checker._names_reassigned_by_read_or_printf(tokens) + assert name in result + assert other_name not in result + + +def test_segment_references_a_name_finds_a_bare_reference() -> None: + """`_segment_references_a_name` recognizes a plain `$NAME` reference + to a poisoned name.""" + assert checker._segment_references_a_name(["gh", "api", "repos/x/y", "-X", "$M"], {"M"}) is True + + +def test_segment_references_a_name_finds_a_braced_reference() -> None: + assert checker._segment_references_a_name(["echo", "${M}"], {"M"}) is True + + +def test_segment_references_a_name_ignores_an_unrelated_reference() -> None: + """No false positive: a reference to a name that is NOT in the + poisoned set is ignored.""" + assert checker._segment_references_a_name(["echo", "$OTHER"], {"M"}) is False + + +def test_segment_references_a_name_empty_names_is_always_false() -> None: + """No poisoned names at all -- the common case for every pre-existing + call site -- never reports a reference, regardless of segment + content.""" + assert checker._segment_references_a_name(["gh", "api", "repos/x/y", "-X", "$M"], set()) is False + + +def test_rule_gh_api_write_denies_a_reference_to_a_poisoned_name() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-sixth round (issue #1375): `_rule_gh_api_write` used to + consume only NAME_TO_VALUE/NAME_TO_RAW_VALUE, both of which stay + STALE across a `read`/`printf -v`/array-element reassignment -- so a + `-X $M` write-method flag resolved through a poisoned M silently read + as the name's OLD, pre-reassignment value. Confirmed live: `M=GET; + read M <<< "POST"; gh api repos/x/y/issues -X $M` classified as "no + denied pattern matched" before this fix.""" + segments = [["gh", "api", "repos/x/y/issues", "-X", "$M"]] + reason = checker._rule_gh_api_write(segments, "gh api repos/x/y/issues -x $m", {"m": "get"}, {}, {"M"}) + assert reason is not None + assert "reassigned" in reason + + +def test_rule_gh_api_write_ignores_a_poisoned_name_outside_a_gh_api_segment() -> None: + """No false positive: a poisoned name referenced in a segment that is + not itself a `gh api` call must not be flagged by this rule.""" + segments = [["echo", "$M"]] + reason = checker._rule_gh_api_write(segments, "echo $m", {}, {}, {"M"}) + assert reason is None + + +def test_rule_gh_api_write_default_poisoned_names_is_empty() -> None: + """Every pre-existing call site (this module's own test file included) + omits NAMES_WITH_DYNAMIC_ASSIGNMENT entirely -- confirm the default + is treated as empty, not as a mysterious universal deny.""" + segments = [["gh", "api", "repos/x/y/issues", "-X", "$M"]] + reason = checker._rule_gh_api_write(segments, "gh api repos/x/y/issues -x $m", {"m": "get"}, {}) + assert reason is None + + +def test_segment_loop_hit_denies_a_dynamic_command_word_referencing_a_poisoned_name() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-sixth round (issue #1375): B1a/B1b's own tool+verb + resolution reads NAME_TO_VALUE/NAME_TO_RAW_VALUE, both stale across a + `read`/`printf -v`/array-element reassignment. Confirmed live via a + stand-in `uv` binary on PATH that `A=harmless; read A <<< "uv"; + B=harmless2; read B <<< "install"; $A $B foo` genuinely runs `uv + install foo` (captured argv: "install foo").""" + segments = [["$A", "$B", "foo"]] + reason, _ = checker._segment_loop_hit(segments, {"a": "harmless", "b": "harmless2"}, {}, {"A", "B"}) + assert reason is not None + assert "reassigned" in reason + + +def test_segment_loop_hit_ignores_a_poisoned_name_in_a_literal_command_word_segment() -> None: + """No false positive: B1a/B1b's own precondition (`seg[0]` itself + dynamic) still gates this poisoning check -- a poisoned name merely + referenced in an otherwise-ordinary, literal-command-word segment + (`echo $M`) must not be flagged, since B1a/B1b would never have fired + on that shape regardless of M's value.""" + segments = [["echo", "$M"]] + reason, _ = checker._segment_loop_hit(segments, {}, {}, {"M"}) + assert reason is None + + +def test_segment_loop_hit_default_poisoned_names_is_empty() -> None: + """Every pre-existing call site omits NAMES_WITH_DYNAMIC_ASSIGNMENT + entirely -- confirm the default is treated as empty.""" + segments = [["$A", "$B", "foo"]] + reason, _ = checker._segment_loop_hit(segments, {"a": "harmless", "b": "harmless2"}, {}) + assert reason is None + + +def test_classify_denies_a_checkout_path_reassigned_via_read() -> None: + """End-to-end regression pin for the round-26 `read`-reassignment + finding at the `classify()` level. Confirmed live before this fix: + `DIR=sub; read DIR <<< "other"; git checkout -- $DIR` resolved + `checkout_restore_paths` to `('sub',)` -- the STALE, pre-reassignment + value (real bash: `$DIR` genuinely becomes "other").""" + verdict = checker.classify('DIR=sub; read DIR <<< "other"; git checkout -- $DIR') + assert verdict.deny is True + assert "cannot be soundly trusted" in verdict.reason + + +def test_classify_denies_a_restore_path_reassigned_via_read() -> None: + """Companion to the checkout pin above, for `git restore`.""" + verdict = checker.classify('DIR=sub; read DIR <<< "other"; git restore $DIR') + assert verdict.deny is True + assert "cannot be soundly trusted" in verdict.reason + + +def test_classify_denies_a_checkout_path_reassigned_via_array_element() -> None: + """End-to-end regression pin for the round-26 array-element-assignment + finding at the `classify()` level. Confirmed live before this fix: + `arr=x; arr[0]=other; git checkout -- $arr` resolved `checkout_ + restore_paths` to `('x',)` -- the STALE, pre-reassignment value (real + bash: `$arr` genuinely becomes "other").""" + verdict = checker.classify("arr=x; arr[0]=other; git checkout -- $arr") + assert verdict.deny is True + assert "cannot be soundly trusted" in verdict.reason + + +def test_classify_denies_a_checkout_path_reassigned_via_printf_v() -> None: + """End-to-end regression pin for the round-26 `printf -v` finding at + the `classify()` level. Confirmed live before this fix: `DIR=sub; + printf -v DIR "%s" other; git checkout -- $DIR` resolved `checkout_ + restore_paths` to `('sub',)` -- the STALE, pre-reassignment value + (real bash: `$DIR` genuinely becomes "other").""" + verdict = checker.classify('DIR=sub; printf -v DIR "%s" other; git checkout -- $DIR') + assert verdict.deny is True + assert "cannot be soundly trusted" in verdict.reason + + +def test_classify_denies_a_gh_api_write_method_reassigned_via_read() -> None: + """End-to-end regression pin for the round-26 `read`-reassignment + finding against `_rule_gh_api_write`. Confirmed live before this fix: + `M=GET; read M <<< "POST"; gh api repos/x/y/issues -X $M` classified + as "no denied pattern matched" -- a full HARD-DENY bypass.""" + verdict = checker.classify('M=GET; read M <<< "POST"; gh api repos/x/y/issues -X $M') + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_write_method_reassigned_via_array_element() -> None: + """Companion to the `read` pin above, for array-element assignment. + Confirmed live before this fix: `M=GET; M[0]=POST; gh api + repos/x/y/issues -X $M` classified as "no denied pattern matched".""" + verdict = checker.classify("M=GET; M[0]=POST; gh api repos/x/y/issues -X $M") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_via_read() -> None: + """End-to-end regression pin for the round-26 `read`-reassignment + finding against B1b (`_rule_b1b_dynamic_word_assigned_tool_and_verb`). + Confirmed live before this fix via a stand-in `uv` binary on PATH: + `A=harmless; read A <<< "uv"; B=harmless2; read B <<< "install"; $A + $B foo` genuinely runs `uv install foo`, but classified as allowed.""" + verdict = checker.classify('A=harmless; read A <<< "uv"; B=harmless2; read B <<< "install"; $A $B foo') + assert verdict.deny is True + + +def test_classify_leaves_a_read_of_an_unrelated_name_unaffected() -> None: + """No false positive: `read`-ing into a name never referenced by a + checkout/restore path, a `gh api` write flag, or a dynamic command + word must not affect classification at all.""" + verdict = checker.classify('read NAME <<< "value"; echo $NAME; git checkout -- sub/file.txt') + assert verdict.deny is False + assert verdict.checkout_restore_paths == ("sub/file.txt",) + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 0ac32f8692c4bb959055d1d4cdb50089e2271bd5 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 00:01:02 +0000 Subject: [PATCH 32/46] fix(hooks): protect gh-api-write/B1a-B1b against a static-then-dynamic reassignment A fresh, independent adversarial review of this PR's current head (round 27) found that round 26's own fix over-corrected: it excluded the ENTIRE round-24 dynamic-RHS-reassignment class and round-25 append class from `_rule_gh_api_write` and `_segment_loop_hit` (B1a/B1b), not just the genuinely-unresolvable sub-case that actually caused the `graphql-mutation-keyword-variable-concatenation` regression -- leaving those two HARD-DENY paths with no protection at all against a name that carries an earlier STATIC value and is LATER dynamically reassigned. Independently reproduced live before acting, via `classify()` and real bash execution with stand-in `uv`/`gh` binaries on PATH: `TOOL=uv; VERB=harmless; VERB=$(echo install); $TOOL $VERB foo` classified as "no denied pattern matched" even though real bash genuinely runs `uv install foo` (captured argv: "install foo"); the append variant (`VERB=inst; VERB+=all`) reproduces identically; `M=safe; M=$(echo POST); gh api repos/o/r/pulls/1/merge -X $M` also wrongly allowed, real bash genuinely running the write call (captured argv confirms `-X POST`). Confirmed through the real wrapper: both `uv` cases and the `gh api` case denied with exit 2 before this fix would have allowed them with exit 0. Closed by a new `_names_reassigned_from_a_static_value` function that draws the distinction round 26's own docstring had already identified but not implemented: a name assigned ONLY dynamically (no earlier static value, e.g. the graphql-mutation case's own `Q`) is genuinely unresolvable and already handled soundly by these two consumers' own existing `_substitute_var_refs_candidates`-based resolution ("no evidence of a write," the deliberately-accepted posture for that specific, disclosed gap); a name WITH an earlier static value that is later dynamically reassigned goes from trustworthy to stale, the same order-blind-collapse defect class every prior round in this file already closes for its own consumer. A new `_names_poisoned_for_gh_ api_and_b1` combines this new function with the existing append detection (factored out into `_names_appended_to`, shared with `_names_with_dynamic_assignment`) and round 26's own untracked- construct detection into the actual set `_rule_gh_api_write`/`_segment_ loop_hit` now consume, replacing round 26's own narrower `_names_ reassigned_by_untracked_construct` at that call site (which remains, unchanged, as one component of the new union). Regression tests added at every established layer: end-to-end wrapper- level for B1a/B1b (plain dynamic reassignment and static append) and gh-api-write against a stand-in `uv` binary (hooks/test_gitapex_check_ bash_safety.py); unit, false-positive, and `@given` Hypothesis property tests for `_names_reassigned_from_a_static_value`, `_names_poisoned_ for_gh_api_and_b1`, and the extracted `_names_appended_to`, plus `classify()`-level end-to-end pins for every reported shape and an explicit regression guard confirming the graphql-mutation known bypass stays allowed (tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI's own whole- codebase invocation), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-27 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 268 ++++++++++++++---- hooks/test_gitapex_check_bash_safety.py | 26 ++ ...st_gitapex_check_bash_safety_properties.py | 137 +++++++++ 3 files changed, 370 insertions(+), 61 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index f120d7cc..994a9180 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3500,9 +3500,10 @@ def _segment_references_a_name(seg: list[str], names: set[str]) -> bool: _POISONED_REASSIGNMENT_GH_API_HIT = ( - "a 'gh api' call references a variable whose value was reassigned via a construct this classifier " - "cannot track (`read`/`readarray`/`mapfile`, `printf -v`, or an array-element assignment) -- rewrite " - "as a plain literal command so it can be checked" + "a 'gh api' call references a variable that was reassigned after an earlier value in a way this " + "classifier cannot soundly resolve (a compound `+=` append, a dynamic reassignment after an " + "earlier static value, or `read`/`readarray`/`mapfile`/`printf -v`/an array-element assignment) " + "-- rewrite as a plain literal command so it can be checked" ) @@ -3511,7 +3512,7 @@ def _rule_gh_api_write( lowered_command: str, name_to_value: dict[str, str], name_to_raw_value: dict[str, str], - names_reassigned_by_untracked_construct: set[str] | None = None, + names_poisoned_for_gh_api_and_b1: set[str] | None = None, ) -> str | None: """`literals` is already lowercased, matching the predecessor script's own case-insensitive match against its whole lowered command -- so @@ -3523,17 +3524,17 @@ def _rule_gh_api_write( (each pass owns its own branching) so this function's own cyclomatic complexity stays low. - NAMES_REASSIGNED_BY_UNTRACKED_CONSTRUCT (round 26, issue #1375) - defaults to `None` (treated as empty) so every pre-existing call site - keeps its exact prior behavior; `_classify_tokens`'s own call sites - are the only ones that supply it, deliberately NOT the same (wider) - NAMES_WITH_DYNAMIC_ASSIGNMENT `_resolve_path_tokens` (checkout/ - restore) consumes -- see `_names_reassigned_by_untracked_construct`'s - own docstring for why. See `_segment_references_a_name`'s own + NAMES_POISONED_FOR_GH_API_AND_B1 (round 26, widened round 27, issue + #1375) defaults to `None` (treated as empty) so every pre-existing + call site keeps its exact prior behavior; `_classify_tokens`'s own + call sites are the only ones that supply it, deliberately NOT the + same (wider) NAMES_WITH_DYNAMIC_ASSIGNMENT `_resolve_path_tokens` + (checkout/restore) consumes -- see `_names_poisoned_for_gh_api_and_ + b1`'s own docstring for why. See `_segment_references_a_name`'s own docstring for why a poisoned name is checked here, once has_gh_api is already confirmed, rather than by deleting the name from NAME_TO_ VALUE/NAME_TO_RAW_VALUE.""" - poisoned = names_reassigned_by_untracked_construct or set() + poisoned = names_poisoned_for_gh_api_and_b1 or set() for seg in segments: literals = [t.lower() for t in seg if not _is_dynamic(t)] has_gh_api = any(literals[i : i + 2] == ["gh", "api"] for i in range(len(literals) - 1)) @@ -3842,6 +3843,26 @@ def _multi_valued_names_referenced( } +def _names_appended_to(tokens: list[str]) -> set[str]: + """Every NAME with at least one `NAME+=value` compound/append- + assignment token in TOKENS (via `_APPEND_ASSIGN_RE`), regardless of + whether the appended text is itself static or dynamic -- round 25's + own posture (see `_names_with_dynamic_assignment`'s own round-25 + paragraph): an append always makes the name's own combined value + unrecoverable without genuine execution-order tracking this + classifier does not perform, so it is poisoned unconditionally. + Factored out so both `_names_with_dynamic_assignment` (checkout/ + restore's own full union) and `_names_poisoned_for_gh_api_and_b1` + (round 27, issue #1375) can share the identical append-detection + logic without duplicating it.""" + names: set[str] = set() + for token in tokens: + match = _APPEND_ASSIGN_RE.match(token) + if match: + names.add(match.group(1)) + return names + + def _names_with_dynamic_assignment(tokens: list[str]) -> set[str]: """Every NAME whose own value, at some point in TOKENS, is genuinely UNKNOWN to this classifier -- either a DYNAMIC value (containing @@ -3958,13 +3979,26 @@ def _names_with_dynamic_assignment(tokens: list[str]) -> set[str]: `candidates is None: True` vs. `candidates == []: False` branches before writing any code -- would make those two consumers LESS safe (an empty candidate list reads as "vacuously safe," the opposite of - the fail-closed direction this fix needs).""" - names: set[str] = set() + the fail-closed direction this fix needs). + + CORRECTION (round 27, issue #1375): the round-26 paragraph above + overstated the exclusion's own precision. Round 26 excluded the + ENTIRE round-24 dynamic-RHS-reassignment class from `_rule_gh_api_ + write`/`_segment_loop_hit`, when only the NARROWER sub-case actually + needed excluding: a name that is ONLY EVER assigned dynamically (no + earlier static value at all, e.g. `Q` above) is genuinely + unresolvable and already handled soundly by those rules' own + existing `_substitute_var_refs_candidates`-based resolution; a name + that carries an EARLIER STATIC value and is LATER dynamically + reassigned (e.g. `VERB=harmless; VERB=$(echo install)`) is a + genuine, confidently-wrong-value reassignment that round 26's own + blanket exclusion left completely unprotected in those two + consumers -- see `_names_reassigned_from_a_static_value`'s own + docstring for the live bypass this closes and + `_names_poisoned_for_gh_api_and_b1`'s own docstring for the actual, + corrected set those two consumers now use.""" + names = _names_appended_to(tokens) for token in tokens: - append_match = _APPEND_ASSIGN_RE.match(token) - if append_match: - names.add(append_match.group(1)) - continue if not _is_dynamic(token): continue match = _ASSIGN_RE.match(token) @@ -3983,18 +4017,28 @@ def _names_reassigned_by_untracked_construct(tokens: list[str]) -> set[str]: `printf -v` reassignment (via `_names_reassigned_by_read_or_printf`). Deliberately NARROWER than `_names_with_dynamic_assignment`'s own - full union -- this is the set `_rule_gh_api_write`/`_segment_loop_ - hit` (B1a/B1b) actually consume, NOT the full one `_resolve_path_ - tokens` (checkout/restore) consumes -- see `_names_with_dynamic_ - assignment`'s own round-26 paragraph for why threading the FULL union - into those two HARD-DENY consumers regressed an existing, unrelated, - deliberately-disclosed bypass. Computed fresh from TOKENS only (no - outer-scope union, unlike `_names_with_dynamic_assignment`'s own - recursive threading) -- a `read`/array-element reassignment occurring - OUTSIDE a `$(...)`/array-literal span but referenced only INSIDE it - is a disclosed, narrower residual this pass does not cover; every - round-26 finding actually verified live was a flat, top-level case, - which this fully covers.""" + full union -- NOT the full one `_resolve_path_tokens` (checkout/ + restore) consumes -- see `_names_with_dynamic_assignment`'s own + round-26 paragraph for why threading the FULL union into `_rule_gh_ + api_write`/`_segment_loop_hit` (B1a/B1b) regressed an existing, + unrelated, deliberately-disclosed bypass. Computed fresh from TOKENS + only (no outer-scope union, unlike `_names_with_dynamic_assignment`'s + own recursive threading) -- a `read`/array-element reassignment + occurring OUTSIDE a `$(...)`/array-literal span but referenced only + INSIDE it is a disclosed, narrower residual this pass does not + cover; every round-26 finding actually verified live was a flat, + top-level case, which this fully covers. + + CORRECTION (round 27, issue #1375): this function alone is no longer + the set `_rule_gh_api_write`/`_segment_loop_hit` actually consume -- + round 26's own claim that it was turned out to be too narrow, since + excluding the round-24/25 classes ENTIRELY (not just their genuinely + unresolvable sub-case) left those two consumers unprotected against + a real reassignment. See `_names_poisoned_for_gh_api_and_b1`'s own + docstring for the actual, corrected set those two consumers now + use -- this function remains one component of that union, still + responsible for exactly the array-element/read/printf-v shapes its + own name and docstring describe.""" names: set[str] = set() for token in tokens: array_match = _ARRAY_ELEMENT_ASSIGN_RE.match(token) @@ -4004,6 +4048,108 @@ def _names_reassigned_by_untracked_construct(tokens: list[str]) -> set[str]: return names +def _names_reassigned_from_a_static_value(tokens: list[str]) -> set[str]: + """Every NAME that carried a trustworthy STATIC value at some point + in TOKENS and was LATER reassigned a DYNAMIC (`$`/backtick- + containing) value via a plain `NAME=value` assignment -- a genuine + reassignment that leaves the name's earlier static value stale and + CONFIDENTLY WRONG, not merely unknown. + + CRITICAL bug found by independent adversarial review (round 27, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: round 26's own `_names_reassigned_by_untracked_ + construct` deliberately excludes the round-24 dynamic-RHS- + reassignment class ENTIRELY from `_rule_gh_api_write`/`_segment_ + loop_hit` (B1a/B1b) -- correctly avoiding a regression of the + `graphql-mutation-keyword-variable-concatenation` known bypass + (`Q="${A}${B} { x }"`, where Q never has an earlier static value at + all) -- but that exclusion was coarser than it needed to be, and + also silently dropped protection for the genuinely dangerous case + where a name DOES carry an earlier static value that a later + dynamic reassignment makes stale. `TOOL=uv; VERB=harmless; + VERB=$(echo install); $TOOL $VERB foo` resolved to `deny=False, + reason='no denied pattern matched'` even though real bash genuinely + runs `uv install foo` (confirmed live via a stand-in `uv` binary on + PATH: captured argv `uv called with: install foo`) -- VERB's own + STALE static value (`harmless`) stayed trusted in `_assigned_raw_ + values` (which silently skips a dynamic-RHS token, per that + function's own docstring) instead of VERB losing all confidence the + way checkout/restore's own `_names_with_dynamic_assignment` already + treats it. A static append variant (`VERB=inst; VERB+=all`) + reproduces identically (captured argv: `uv called with: install + foo`) -- already covered by `_names_appended_to`, not this + function, cited here only as confirmation the two gaps compound. + Reproduced identically for `_rule_gh_api_write`: `M=safe; + M=$(echo POST); gh api repos/o/r/pulls/1/merge -X $M` also resolved + to `deny=False` -- real bash genuinely runs `gh api + repos/o/r/pulls/1/merge -X POST` (captured argv confirms it), a + genuine unreviewed write (e.g. merging a pull request). + + The distinguishing test that safely separates this case from the + graphql-mutation one, so this function can close the gap above + without reopening that regression: a name with NO earlier static + assignment was never trustworthy to begin with (unresolvable, so + gh-api-write's own existing `_substitute_var_refs_candidates`-based + resolution already reads it as "no evidence of a write" -- the + deliberately-accepted posture for that specific, disclosed gap); a + name WITH an earlier static assignment that is later dynamically + reassigned goes from trustworthy to stale, which is exactly the + order-blind-collapse defect class every other round in this file's + own history (19-26) already closes for its own specific consumer -- + this function closes it here too, without reopening the graphql- + mutation regression, because it only fires when EVER_STATIC already + recorded a real prior static assignment for that exact name.""" + ever_static: set[str] = set() + poisoned: set[str] = set() + for token in tokens: + match = _ASSIGN_RE.match(token) + if not match: + continue + name = match.group(1) + if _is_dynamic(token): + if name in ever_static: + poisoned.add(name) + else: + ever_static.add(name) + return poisoned + + +def _names_poisoned_for_gh_api_and_b1(tokens: list[str]) -> set[str]: + """The full poisoned-names set `_rule_gh_api_write` and `_segment_ + loop_hit` (B1a/B1b) actually consume -- the union of every class of + reassignment that makes a name's own recorded value stale or + unrecoverable for those two consumers specifically: + + - `_names_appended_to`: any `NAME+=value` compound-assignment + (round 25), poisoned unconditionally regardless of whether the + appended text is static or dynamic. + - `_names_reassigned_from_a_static_value`: a name reassigned a + dynamic value AFTER an earlier static one (round 27's own + narrowing of the round-24 class -- see that function's own + docstring for the live bypass this restores and why it does not + reopen the graphql-mutation regression). + - `_names_reassigned_by_untracked_construct`: array-element + assignment and `read`/`readarray`/`mapfile`/`printf -v` + reassignment (round 26), constructs `_ASSIGN_RE`/`_APPEND_ + ASSIGN_RE` never recognize at all. + + Deliberately EXCLUDES a name that is assigned ONLY dynamically, with + no earlier static value anywhere in TOKENS -- that case is the + `graphql-mutation-keyword-variable-concatenation` known bypass's own + shape, already handled soundly by these two consumers' own existing + `_substitute_var_refs_candidates`-based resolution (an unresolvable + candidate list reads as "no evidence of a write," the deliberately- + accepted posture for that specific, disclosed gap) -- poisoning it + here would regress that case into a false-positive deny, exactly + what round 26's own (too-broad) exclusion was trying to prevent.""" + return ( + _names_appended_to(tokens) + | _names_reassigned_from_a_static_value(tokens) + | _names_reassigned_by_untracked_construct(tokens) + ) + + def _names_reassigned_by_read_or_printf(tokens: list[str]) -> set[str]: """Every NAME that `read`/`readarray`/`mapfile` (any bare-identifier- shaped operand following the command word, skipping option flags, @@ -5286,7 +5432,7 @@ def _segment_loop_hit( segments: list[list[str]], name_to_value: dict[str, str], name_to_raw_value: dict[str, str], - names_reassigned_by_untracked_construct: set[str] | None = None, + names_poisoned_for_gh_api_and_b1: set[str] | None = None, ) -> tuple[str | None, bool]: """The B1a/B1b/B2/obfuscated-git-push-second-token loop -- factored out of `_classify_tokens` so it can be run TWICE: once against @@ -5313,30 +5459,32 @@ def _segment_loop_hit( is no equivalent gap for that shape in this file specifically -- only B2's own literal-`seg[0]` requirement is affected. - NAMES_REASSIGNED_BY_UNTRACKED_CONSTRUCT (round 26, issue #1375) - defaults to `None` (treated as empty) so every pre-existing call site - keeps its exact prior behavior; `_classify_tokens`'s own call sites - are the only ones that supply it, deliberately NOT the same (wider) - NAMES_WITH_DYNAMIC_ASSIGNMENT `_resolve_path_tokens` (checkout/ - restore) consumes -- see `_names_reassigned_by_untracked_construct`'s - own docstring for why. Checked ONLY when `seg[0]` is itself dynamic - -- the same precondition B1a/B1b already require before either even - runs (see each rule's own docstring) -- so a poisoned name referenced - in an otherwise-harmless, literal-command-word segment (e.g. `echo - $M` where M was reassigned via `read`) is never flagged here: B1a/ - B1b's own bypass only exists where a poisoned name feeds the SAME - position (the dynamically-constructed command word itself, or a - same-segment verb token) those two rules already resolve, so this - check is scoped identically rather than treating every reference - anywhere as a hit.""" + NAMES_POISONED_FOR_GH_API_AND_B1 (round 26, widened round 27, issue + #1375) defaults to `None` (treated as empty) so every pre-existing + call site keeps its exact prior behavior; `_classify_tokens`'s own + call sites are the only ones that supply it, deliberately NOT the + same (wider) NAMES_WITH_DYNAMIC_ASSIGNMENT `_resolve_path_tokens` + (checkout/restore) consumes -- see `_names_poisoned_for_gh_api_and_ + b1`'s own docstring for why. Checked ONLY when `seg[0]` is itself + dynamic -- the same precondition B1a/B1b already require before + either even runs (see each rule's own docstring) -- so a poisoned + name referenced in an otherwise-harmless, literal-command-word + segment (e.g. `echo $M` where M was reassigned via `read`) is never + flagged here: B1a/B1b's own bypass only exists where a poisoned name + feeds the SAME position (the dynamically-constructed command word + itself, or a same-segment verb token) those two rules already + resolve, so this check is scoped identically rather than treating + every reference anywhere as a hit.""" is_git_push = False - poisoned = names_reassigned_by_untracked_construct or set() + poisoned = names_poisoned_for_gh_api_and_b1 or set() for seg in segments: if seg and _is_dynamic(seg[0]) and _segment_references_a_name(seg, poisoned): return ( - "a Bash command word is dynamically constructed from a variable whose value was " - "reassigned via a construct this classifier cannot track (`read`, `printf -v`, or an " - "array-element assignment) -- rewrite as a plain literal command so it can be checked", + "a Bash command word is dynamically constructed from a variable that was reassigned " + "after an earlier value in a way this classifier cannot soundly resolve (a compound " + "`+=` append, a dynamic reassignment after an earlier static value, or `read`/" + "`printf -v`/an array-element assignment) -- rewrite as a plain literal command so it " + "can be checked", is_git_push, ) if _rule_b1a_dynamic_word_same_segment_verb(seg, _WATCHED_VERBS, name_to_value, name_to_raw_value): @@ -5600,10 +5748,10 @@ def _classify_tokens( names_with_dynamic_assignment = outer_dynamic_names | _names_with_dynamic_assignment(tokens) # Deliberately narrower than NAMES_WITH_DYNAMIC_ASSIGNMENT above, and # computed fresh from TOKENS only (no outer-scope union) -- see - # `_names_reassigned_by_untracked_construct`'s own docstring for why + # `_names_poisoned_for_gh_api_and_b1`'s own docstring for why # `_rule_gh_api_write`/`_segment_loop_hit` below consume THIS set, # not the full one just above. - names_reassigned_by_untracked_construct = _names_reassigned_by_untracked_construct(tokens) + names_poisoned_for_gh_api_and_b1 = _names_poisoned_for_gh_api_and_b1(tokens) lowered_command = " ".join(tokens).lower() is_git_push = is_git_push or any(_is_git_push_segment(seg, raw_assigned) for seg in segments) @@ -5613,24 +5761,22 @@ def _classify_tokens( return Verdict(True, literal_hit, is_git_push, checkout_restore_paths) gh_api_hit = _rule_gh_api_write( - segments, lowered_command, assigned, raw_assigned, names_reassigned_by_untracked_construct + segments, lowered_command, assigned, raw_assigned, names_poisoned_for_gh_api_and_b1 ) or _rule_gh_api_write( segments, lowered_command, assigned_write_biased, raw_assigned_write_biased, - names_reassigned_by_untracked_construct, + names_poisoned_for_gh_api_and_b1, ) if gh_api_hit: return Verdict(True, gh_api_hit, is_git_push, checkout_restore_paths) - loop_hit, loop_is_git_push = _segment_loop_hit( - segments, assigned, raw_assigned, names_reassigned_by_untracked_construct - ) + loop_hit, loop_is_git_push = _segment_loop_hit(segments, assigned, raw_assigned, names_poisoned_for_gh_api_and_b1) is_git_push = is_git_push or loop_is_git_push if not loop_hit: loop_hit, biased_loop_is_git_push = _segment_loop_hit( - segments, assigned_write_biased, raw_assigned_write_biased, names_reassigned_by_untracked_construct + segments, assigned_write_biased, raw_assigned_write_biased, names_poisoned_for_gh_api_and_b1 ) is_git_push = is_git_push or biased_loop_is_git_push if loop_hit: @@ -5641,7 +5787,7 @@ def _classify_tokens( ] if collapsed_segments != segments: collapsed_hit, collapsed_is_git_push = _segment_loop_hit( - collapsed_segments, assigned, raw_assigned, names_reassigned_by_untracked_construct + collapsed_segments, assigned, raw_assigned, names_poisoned_for_gh_api_and_b1 ) is_git_push = is_git_push or collapsed_is_git_push if not collapsed_hit: @@ -5649,7 +5795,7 @@ def _classify_tokens( collapsed_segments, assigned_write_biased, raw_assigned_write_biased, - names_reassigned_by_untracked_construct, + names_poisoned_for_gh_api_and_b1, ) is_git_push = is_git_push or collapsed_biased_is_git_push if collapsed_hit: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index d0a081dc..39863f68 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -939,6 +939,32 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: 'M=GET; printf -v M "%s" POST; gh api repos/o/r/pulls/1/merge -X $M', "gh-api-method-value-reassigned-via-printf-v", ), + # Found live by Step 8 independent review, twenty-seventh round + # (issue #1375): round 26's own fix (the two `_names_reassigned_by_ + # untracked_construct` cases just above) deliberately excluded the + # round-24 plain-dynamic-reassignment and round-25 append classes + # ENTIRELY from B1a/B1b and gh-api-write, not just their genuinely- + # unresolvable sub-case -- leaving a name with an earlier STATIC + # value, later reassigned dynamically, with NO protection at all. + # Confirmed live via a stand-in `uv` binary on PATH (captured argv: + # "install foo") that `TOOL=uv; VERB=harmless; VERB=$(echo install); + # $TOOL $VERB foo` genuinely runs `uv install foo`. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value", + ), + # Same round, the append counterpart. + ( + "TOOL=uv; VERB=inst; VERB+=all; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-via-static-append", + ), + # Same round, the gh-api-write counterpart: real bash genuinely ran + # `gh api repos/o/r/pulls/1/merge -X POST` (captured argv confirms + # it), a genuine, unreviewed write API call. + ( + "M=safe; M=$(echo POST); gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 56b907c0..cdb4e002 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2503,6 +2503,143 @@ def test_classify_leaves_a_read_of_an_unrelated_name_unaffected() -> None: assert verdict.checkout_restore_paths == ("sub/file.txt",) +def test_names_reassigned_from_a_static_value_finds_a_static_then_dynamic_reassignment() -> None: + """Regression pin for the real bypass found live by Step 8 independent + review, twenty-seventh round (issue #1375): round 26's own + `_names_reassigned_by_untracked_construct` deliberately excluded the + ENTIRE round-24 dynamic-RHS-reassignment class from `_rule_gh_api_ + write`/`_segment_loop_hit`, not just its genuinely-unresolvable + sub-case -- leaving a name with an earlier STATIC value, later + reassigned dynamically, with no protection at all in those two + consumers. This function restores exactly that narrower case.""" + assert checker._names_reassigned_from_a_static_value(["VERB=harmless", "VERB=$(echo install)"]) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_ignores_a_dynamic_only_name() -> None: + """No false positive, and the exact distinction that keeps the + `graphql-mutation-keyword-variable-concatenation` known bypass + (`Q="${A}${B} { x }"`, Q never has an earlier static value) from + regressing: a name assigned ONLY dynamically, with no earlier static + value anywhere, is not poisoned by this function.""" + assert checker._names_reassigned_from_a_static_value(["Q=$A$B"]) == set() + + +def test_names_reassigned_from_a_static_value_ignores_a_static_only_name() -> None: + """No false positive: a name assigned only static values anywhere is + not flagged, matching every other reassignment-detector in this + module.""" + assert checker._names_reassigned_from_a_static_value(["VERB=harmless", "VERB=other"]) == set() + + +def test_names_reassigned_from_a_static_value_ignores_an_unrelated_name() -> None: + """No false positive: a static-then-dynamic reassignment of one name + does not poison a completely different name.""" + result = checker._names_reassigned_from_a_static_value(["DIR=sub", "VERB=harmless", "VERB=$(echo install)"]) + assert result == {"VERB"} + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES) +def test_names_reassigned_from_a_static_value_matches_model(name: str, static_value: str, dynamic_value: str) -> None: + """Model-based: for ANY identifier assigned a static value and then + reassigned a dynamic one, `_names_reassigned_from_a_static_value` + always includes it -- and a second, entirely unrelated identifier + that is only ever assigned dynamically (no earlier static value) is + never included alongside it.""" + other_name = name + "_OTHER" + tokens = [f"{name}={static_value}", f"{name}=$({dynamic_value})", f"{other_name}=$({dynamic_value})"] + result = checker._names_reassigned_from_a_static_value(tokens) + assert name in result + assert other_name not in result + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, appended_value=_VALUES) +def test_names_appended_to_matches_model(name: str, static_value: str, appended_value: str) -> None: + """Model-based, exercising the round-27-extracted `_names_appended_ + to` helper directly (round 25's own append-detection logic, factored + out so `_names_with_dynamic_assignment` and `_names_poisoned_for_gh_ + api_and_b1` can share it): for ANY identifier assigned a static value + and then appended to (`+=`), `_names_appended_to` always includes it + -- and a second, entirely unrelated identifier that is only ever + assigned statically is never included alongside it.""" + other_name = name + "_OTHER" + tokens = [f"{name}={static_value}", f"{name}+=$({appended_value})", f"{other_name}={static_value}"] + result = checker._names_appended_to(tokens) + assert name in result + assert other_name not in result + + +def test_names_poisoned_for_gh_api_and_b1_unions_all_three_components() -> None: + """`_names_poisoned_for_gh_api_and_b1` is the union of append + (round 25), static-then-dynamic reassignment (round 27), and + untracked-construct (round 26) poisoning -- confirm all three + classes are actually reachable through the single combined + function `_rule_gh_api_write`/`_segment_loop_hit` consume.""" + tokens = [ + "APPENDED=x", + "APPENDED+=y", + "STATIC_THEN_DYNAMIC=harmless", + "STATIC_THEN_DYNAMIC=$(echo install)", + "arr=x", + "arr[0]=other", + ] + result = checker._names_poisoned_for_gh_api_and_b1(tokens) + assert result == {"APPENDED", "STATIC_THEN_DYNAMIC", "arr"} + + +def test_names_poisoned_for_gh_api_and_b1_excludes_a_dynamic_only_name() -> None: + """The whole point of round 27's narrowing: a name assigned ONLY + dynamically must still be excluded from the combined poisoned set, + exactly as `_names_reassigned_from_a_static_value` excludes it on + its own -- this is what keeps the graphql-mutation known bypass + from regressing.""" + assert checker._names_poisoned_for_gh_api_and_b1(["Q=$A$B"]) == set() + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_via_plain_dynamic_reassignment() -> None: + """End-to-end regression pin for the round-27 finding at the + `classify()` level, against B1b. Confirmed live before this fix via + a stand-in `uv` binary on PATH: `TOOL=uv; VERB=harmless; + VERB=$(echo install); $TOOL $VERB foo` genuinely runs `uv install + foo` (captured argv: "install foo"), but classified as allowed + ('no denied pattern matched') -- VERB's stale static value + ('harmless') stayed trusted in `_assigned_raw_values`.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_via_a_static_append() -> None: + """Companion to the plain-dynamic-reassignment pin above, for the + append shape. Confirmed live before this fix via a stand-in `uv` + binary on PATH: `TOOL=uv; VERB=inst; VERB+=all; $TOOL $VERB foo` + genuinely runs `uv install foo` (captured argv: "install foo").""" + verdict = checker.classify("TOOL=uv; VERB=inst; VERB+=all; $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_write_method_reassigned_via_plain_dynamic_reassignment() -> None: + """Companion to the B1b pins above, for `_rule_gh_api_write`. + Confirmed live before this fix via a stand-in `gh` binary on PATH: + `M=safe; M=$(echo POST); gh api repos/o/r/pulls/1/merge -X $M` + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST` (a real, + unreviewed write, e.g. merging a pull request), but classified as + allowed.""" + verdict = checker.classify("M=safe; M=$(echo POST); gh api repos/o/r/pulls/1/merge -X $M") + assert verdict.deny is True + + +def test_classify_leaves_the_graphql_mutation_known_bypass_unaffected() -> None: + """Regression guard: the round-27 fix must NOT reopen the pre- + existing, deliberately-disclosed `graphql-mutation-keyword- + variable-concatenation` known bypass (`KNOWN_BYPASS_COMMANDS`) -- + `Q` here is assigned ONLY dynamically, with no earlier static value, + so it must stay outside the poisoned set and this command must stay + allowed exactly as before round 27.""" + verdict = checker.classify('A=muta; B=tion; Q="${A}${B} { x }"; gh api graphql -f query="$Q"') + assert verdict.deny is False + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 412ad482a30ef6966db00ff7977b718dc228b3fe Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 00:21:15 +0000 Subject: [PATCH 33/46] fix(hooks): clear a static-value poisoning once a later static reassignment restores it A fresh, independent adversarial review of this PR's current head (round 28) found that round 27's own `_names_reassigned_from_a_static_ value` only ever ADDED a name to its own poisoned set, never removed it -- so a name reassigned static -> dynamic -> static again (a THIRD assignment that fully restores a trustworthy, resolvable value, the genuine value real bash uses at the point of use) stayed poisoned forever. This contradicted the function's own docstring, which already claimed to test "is the name's LATEST assignment dynamic" but actually implemented "was the name EVER dynamically reassigned anywhere" -- a false-positive over-denial of an actually-safe command, not a security bypass. Independently reproduced live before acting, via `classify()` and real bash execution with stand-in `uv`/`gh` binaries: `TOOL=uv; VERB=harmless; VERB=$(echo x); VERB=status; $TOOL $VERB foo` genuinely runs `uv status foo` (`status` is not a watched verb) but was wrongly denied by B1a/B1b; `M=GET; M=$(echo x); M=HEAD; gh api repos/o/r/issues -X $M` genuinely runs `gh api repos/o/r/issues -X HEAD` (a read method) but was wrongly denied by gh-api-write. Confirmed through the real wrapper: both now correctly allow with exit 0. A superficially similar case, `M=POST; M=$(echo x); M=GET; ...`, was checked and confirmed NOT to be a counter-example: `_assigned_raw_ values_biased_toward`'s own independent, deliberate "once a name is assigned a watched-write-method value at ANY point, it stays biased toward that value" posture (round 22's own established, documented design) correctly keeps denying that specific case regardless of this fix, since POST genuinely was assigned at some point and this module's own established posture treats that as reason enough for extra scrutiny -- a real, independently-verified case of intended defense- in-depth, not a residual bug. A dedicated regression test pins this no-under-correction guarantee. Closed by having a later static reassignment clear the name from the poisoned set (`poisoned.discard(name)`, not just skip adding to it): EVER_STATIC still records that a static value was seen at all, so a FURTHER dynamic reassignment after the clearing still correctly re-poisons the name (verified live and via a dedicated regression test) -- the poisoning now genuinely tracks the name's latest assignment, matching the docstring's own original claim. Regression tests added at every established layer: end-to-end wrapper- level for both the false-positive-clearing case and the no-under- correction (write-bias-still-denies) case against a real wrapper and stand-in `uv`/`gh` binaries (hooks/test_gitapex_check_bash_safety.py); unit, false-positive, and `@given` Hypothesis property tests for the clear-then-re-poison sequence, plus `classify()`-level end-to-end pins for both the newly-allowed cases and the still-denied write-bias case (tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI's own whole-codebase invocation), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-28 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 43 ++++++++++- hooks/test_gitapex_check_bash_safety.py | 35 +++++++++ ...st_gitapex_check_bash_safety_properties.py | 77 +++++++++++++++++++ 3 files changed, 154 insertions(+), 1 deletion(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 994a9180..ce919992 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4099,7 +4099,47 @@ def _names_reassigned_from_a_static_value(tokens: list[str]) -> set[str]: own history (19-26) already closes for its own specific consumer -- this function closes it here too, without reopening the graphql- mutation regression, because it only fires when EVER_STATIC already - recorded a real prior static assignment for that exact name.""" + recorded a real prior static assignment for that exact name. + + CRITICAL bug found by independent adversarial review (round 28, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: this function's own round-27 form only ever ADDED to + POISONED, never removed from it -- so a name reassigned static -> + dynamic -> static again (a THIRD assignment that fully restores a + trustworthy, resolvable value, the genuine value real bash uses at + the point of use) stayed poisoned forever, a FALSE-POSITIVE over- + denial of an actually-safe command, contradicting this function's + own docstring above (which already claimed to test "is the name's + LATEST assignment dynamic" but actually implemented "was the name + EVER dynamically reassigned anywhere"). `TOOL=uv; VERB=harmless; + VERB=$(echo x); VERB=status; $TOOL $VERB foo` -- real bash's final + value of `$VERB` is `status`, not a watched verb at all (confirmed + live via a stand-in `uv` binary on PATH: captured argv `uv called + with: status foo`) -- was wrongly denied by B1a/B1b before this fix. + `M=GET; M=$(echo x); M=HEAD; gh api repos/o/r/issues -X $M` + reproduces the identical shape for `_rule_gh_api_write` (both GET + and HEAD are read methods, confirmed live via a stand-in `gh` binary + on PATH: captured argv `gh called with: api repos/o/r/issues -X + HEAD`) -- also wrongly denied before this fix. (A superficially + similar case, `M=POST; M=$(echo x); M=GET; ...`, is NOT a + counter-example to test against: `_assigned_raw_values_biased_ + toward`'s own independent, deliberate "once a name is assigned a + watched-write-method value at ANY point, it stays biased toward that + value" posture -- round 22's own established, documented design, + see that function's own docstring -- correctly keeps denying that + specific case regardless of this fix, since `POST` genuinely was + assigned at some point and this module's own established posture + treats that as reason enough for extra scrutiny; this function's own + fix only concerns names that never carried a watched-write-biased + value at all, like `status`/`HEAD` above.) Closed by having a LATER + static reassignment clear the name from POISONED (`poisoned. + discard(name)`, not just skip adding to it): EVER_STATIC still + records that a static value was seen at all (so a FURTHER dynamic + reassignment after this one still correctly re-poisons the name -- + verified live: `M=GET; M=$(echo x); M=HEAD; M=$(echo other)` leaves + `M` poisoned again, matching real bash's own final, genuinely- + dynamic value).""" ever_static: set[str] = set() poisoned: set[str] = set() for token in tokens: @@ -4112,6 +4152,7 @@ def _names_reassigned_from_a_static_value(tokens: list[str]) -> set[str]: poisoned.add(name) else: ever_static.add(name) + poisoned.discard(name) return poisoned diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 39863f68..1a03a5c3 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -327,6 +327,30 @@ def assert_allowed(command: str) -> None: # OWN command word (`seg[0]`) is itself dynamic -- an unrelated `read` # elsewhere in the command must stay allowed. ('read UNRELATED <<< "x"; echo hello', "read-into-unrelated-name-stays-allowed"), + # False-positive guard for the round-28 fix to + # `_names_reassigned_from_a_static_value` (issue #1375): the + # round-27 form only ever ADDED to its own poisoned set, never + # removed from it, so a name reassigned static -> dynamic -> static + # again (a THIRD assignment fully restoring a trustworthy value) + # stayed poisoned forever. Confirmed live via a stand-in `uv` binary + # on PATH that `TOOL=uv; VERB=harmless; VERB=$(echo x); VERB=status; + # $TOOL $VERB foo` genuinely runs `uv status foo` -- `status` is not + # a watched verb. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo x); VERB=status; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-static-dynamic-static-stays-allowed", + ), + # Same round, the gh-api-write counterpart -- deliberately using a + # read-method pair (GET/HEAD) that never triggers the SEPARATE, + # deliberately sticky write-bias mechanism (round 22), which would + # otherwise mask this specific fix's own effect. Confirmed live via + # a stand-in `gh` binary on PATH that `M=GET; M=$(echo x); M=HEAD; + # gh api repos/o/r/issues -X $M` genuinely runs `gh api + # repos/o/r/issues -X HEAD`, a read method. + ( + "M=GET; M=$(echo x); M=HEAD; gh api repos/o/r/issues -X $M", + "gh-api-method-value-reassigned-static-dynamic-static-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -965,6 +989,17 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "M=safe; M=$(echo POST); gh api repos/o/r/pulls/1/merge -X $M", "gh-api-method-value-reassigned-from-a-static-value", ), + # Round 28's own no-under-correction guard: a name that carries a + # watched write method (POST) at ANY point must stay denied even + # after a later, static reassignment to a read method (GET) -- + # `_assigned_raw_values_biased_toward`'s own independent, sticky + # write-bias mechanism (round 22) must keep denying this regardless + # of round 28's own fix to a SEPARATE mechanism (`_names_reassigned_ + # from_a_static_value`'s own poisoned-set clearing). + ( + "M=POST; M=$(echo x); M=GET; gh api repos/o/r/issues -X $M", + "gh-api-method-value-ever-a-watched-write-method-stays-denied", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index cdb4e002..b4bdfe91 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2538,6 +2538,41 @@ def test_names_reassigned_from_a_static_value_ignores_an_unrelated_name() -> Non assert result == {"VERB"} +def test_names_reassigned_from_a_static_value_clears_on_a_later_static_reassignment() -> None: + """Regression pin for the real false-positive over-denial found live + by Step 8 independent review, twenty-eighth round (issue #1375): + this function's own round-27 form only ever ADDED to POISONED, never + removed from it, so a name reassigned static -> dynamic -> static + again (a THIRD assignment that fully restores a trustworthy, + resolvable value) stayed poisoned forever, contradicting real bash's + own final, genuinely-static value at the point of use.""" + assert checker._names_reassigned_from_a_static_value(["M=GET", "M=$(echo x)", "M=HEAD"]) == set() + + +def test_names_reassigned_from_a_static_value_re_poisons_after_a_further_dynamic_reassignment() -> None: + """No under-correction: a name that IS poisoned, then briefly + cleared by a static reassignment, must be poisoned again by a + FURTHER dynamic reassignment after that -- EVER_STATIC alone must + not permanently disable poisoning for a name once it has ever seen + one static value.""" + tokens = ["M=GET", "M=$(echo x)", "M=HEAD", "M=$(echo other)"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"M"} + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, final_static_value=_VALUES) +def test_names_reassigned_from_a_static_value_matches_model_for_a_static_dynamic_static_sequence( + name: str, static_value: str, dynamic_value: str, final_static_value: str +) -> None: + """Model-based: for ANY identifier reassigned static -> dynamic -> + static again, `_names_reassigned_from_a_static_value` never includes + it -- the final static reassignment is the genuine, trustworthy + value real bash uses at the point of use, and this function must not + poison a name whose latest assignment is static.""" + tokens = [f"{name}={static_value}", f"{name}=$({dynamic_value})", f"{name}={final_static_value}"] + assert name not in checker._names_reassigned_from_a_static_value(tokens) + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model(name: str, static_value: str, dynamic_value: str) -> None: @@ -2640,6 +2675,48 @@ def test_classify_leaves_the_graphql_mutation_known_bypass_unaffected() -> None: assert verdict.deny is False +def test_classify_allows_a_b1b_tool_and_verb_reassigned_from_static_to_dynamic_to_static() -> None: + """End-to-end regression pin for the round-28 finding at the + `classify()` level, against B1b. Confirmed live before this fix via + a stand-in `uv` binary on PATH: `TOOL=uv; VERB=harmless; + VERB=$(echo x); VERB=status; $TOOL $VERB foo` genuinely runs `uv + status foo` (captured argv: "status foo") -- `status` is not a + watched verb -- but was wrongly denied (poisoned forever once ANY + dynamic reassignment was ever seen, regardless of a later static one + fully restoring a trustworthy value).""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo x); VERB=status; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_gh_api_method_reassigned_from_static_to_dynamic_to_static() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`, using a + method pair (GET/HEAD) that never triggers the SEPARATE, + deliberately sticky `_assigned_raw_values_biased_toward` write-bias + mechanism (round 22) -- that mechanism keeps denying a POST-then-GET + sequence regardless of this fix, by its own independent, documented + design (a watched-write-method value seen at ANY point stays sticky), + which is not what this pin is testing. Confirmed live before this + fix via a stand-in `gh` binary on PATH: `M=GET; M=$(echo x); M=HEAD; + gh api repos/o/r/issues -X $M` genuinely runs `gh api + repos/o/r/issues -X HEAD` (captured argv confirms it) -- a read + method -- but was wrongly denied.""" + verdict = checker.classify("M=GET; M=$(echo x); M=HEAD; gh api repos/o/r/issues -X $M") + assert verdict.deny is False + + +def test_classify_still_denies_a_gh_api_method_that_was_ever_a_watched_write_method() -> None: + """No under-correction: the round-28 fix to `_names_reassigned_from_ + a_static_value` must NOT weaken the SEPARATE, pre-existing, + deliberately sticky write-bias mechanism (`_assigned_raw_values_ + biased_toward`, round 22) -- a name that carries a watched write + method (POST) at ANY point stays denied even after a later, static + reassignment to a read method (GET), by that mechanism's own + independent, documented design (extra scrutiny once a dangerous + value is ever seen, regardless of a later overwrite).""" + verdict = checker.classify("M=POST; M=$(echo x); M=GET; gh api repos/o/r/issues -X $M") + assert verdict.deny is True + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From bf3818e987559d46ff61ddcf2d07a2aa615828b5 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 00:46:47 +0000 Subject: [PATCH 34/46] fix(hooks): resolve indirect ${!NAME} references two levels deep for gh-api-write/B1a-B1b A fresh, independent adversarial review of this PR's current head (round 29) found that `_segment_references_a_name`'s own round-26 form deliberately did only a single-level scan for indirect `${!NAME}` references -- its own docstring argued this was sufficient because it only needed to know whether a poisoned name itself was mentioned, not what an indirect reference through it might resolve to. That reasoning was wrong: wrapping an already-poisoned name behind one extra `${!MREF}` indirection layer defeated detection entirely, for every poisoning class this function guards (round 27's static-then-dynamic reassignment, round 25's append, and round 26's own read/array-element/ printf-v classes alike). Independently reproduced live before acting, via `classify()` and real bash execution with stand-in `uv`/`gh` binaries on PATH: `TOOL=uv; VERB=harmless; VERB=$(echo install); MREF=VERB; $TOOL ${!MREF} foo` classified as "no denied pattern matched" even though real bash genuinely runs `uv install foo` (captured argv: "install foo") -- the direct reference (`$TOOL $VERB foo`, no indirection) already correctly denied via round 27's own fix; only the added `${!MREF}` layer defeated it. Reproduced identically for the append and read-reassignment classes, and for `_rule_gh_api_write` (`M=safe; M=$(echo POST); MREF=M; gh api repos/o/r/pulls/1/merge -X ${!MREF}` -- real bash genuinely runs the write call with `-X POST`, a genuine unreviewed write). Confirmed through the real wrapper: all four denied with exit 2 before this fix would have allowed them with exit 0. The sibling checkout/restore consumer (`_referenced_names`, via `_resolve_path_tokens`) was never affected -- it already does the correct two-level resolution this fix now also delegates to. Closed by rewriting `_segment_references_a_name` to delegate directly to the already-correct `_referenced_names` function (added round 24 for checkout/restore's own consumer) instead of maintaining a separate, narrower, buggy single-level scan -- the same reuse-over-re-derivation pattern established by round 27's own `_names_appended_to` extraction. This required threading two new parameters (`name_to_raw_value`, `name_to_raw_value_history`) through `_rule_gh_api_write` and `_segment_loop_hit`'s own signatures and all five call sites in `_classify_tokens`, reusing the already-computed `raw_assigned_history` local variable at each site. Regression tests added at every established layer: end-to-end wrapper-level for the static-then-dynamic, append, and read-reassignment classes (B1a/B1b) and gh-api-write, each referenced indirectly, plus a companion false-positive guard for an indirect reference to an unrelated name despite a genuinely poisoned name elsewhere in scope (hooks/test_gitapex_check_bash_safety.py); unit tests for `_segment_references_a_name`'s new two-level resolution (direct hit, indirect hit, unrelated-indirect no-hit), a `@given` Hypothesis property test modeling the indirect-reference resolution for any identifier pair, and `classify()`-level end-to-end pins for every reported shape plus the false-positive guard (tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format (including the SIM110 simplification ruff itself flagged in the new delegation), mypy, xenon (CI's own whole-codebase invocation), the detection-logic property- coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-29 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 92 +++++++++++++------ hooks/test_gitapex_check_bash_safety.py | 44 +++++++++ ...st_gitapex_check_bash_safety_properties.py | 92 ++++++++++++++++++- 3 files changed, 198 insertions(+), 30 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index ce919992..ab90ec76 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -3468,35 +3468,58 @@ def _gh_api_field_fused_flagname_dynamic_hit( return False -def _segment_references_a_name(seg: list[str], names: set[str]) -> bool: +def _segment_references_a_name( + seg: list[str], + names: set[str], + name_to_raw_value: dict[str, str], + name_to_raw_value_history: dict[str, tuple[str, ...]], +) -> bool: """True when any token in SEG references (as a bare `$NAME`, braced - `${NAME}`, default-clause `${NAME:-x}`, or indirect `${!NAME}` - reference) any name in NAMES. A coarse, single-level scan via the - same `_VAR_REF_FULL_RE` `_referenced_names` uses for its own first - level -- deliberately NOT that function's full two-level indirect - expansion (this only needs to know whether a POISONED name itself is - mentioned, not what an indirect reference through it might resolve - to), so it needs neither NAME_TO_RAW_VALUE nor NAME_TO_RAW_VALUE_ - HISTORY -- both absent from `_rule_gh_api_write`'s and `_segment_ - loop_hit`'s own signatures. + `${NAME}`, default-clause `${NAME:-x}`, or indirect `${!NAME}`, + TWO-level-resolved) any name in NAMES. Delegates directly to + `_referenced_names` (per-token) so this shares that function's own + two-level indirect-reference resolution exactly, rather than + re-deriving a narrower subset of the same logic. Added by round 26 (issue #1375) alongside `_names_with_dynamic_ assignment`'s own extension -- see that function's own docstring for the live `_rule_gh_api_write`/B1a/B1b bypasses this closes, and for why simply deleting a poisoned name's dict entry (the first approach considered) would have made those two consumers LESS safe rather - than more.""" + than more. + + CRITICAL bug found by independent adversarial review (round 29, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: this function's own round-26 form deliberately did + ONLY a single-level scan (its own docstring argued "this only needs + to know whether a POISONED name itself is mentioned, not what an + indirect reference through it might resolve to") -- that reasoning + was WRONG: wrapping an already-poisoned name behind one extra + `${!MREF}` indirection layer defeated detection entirely, for every + poisoning class this function guards (round 27's static-then- + dynamic reassignment, round 25's append, and round 26's own read/ + array-element/printf-v classes alike). `TOOL=uv; VERB=harmless; + VERB=$(echo install); MREF=VERB; $TOOL ${!MREF} foo` resolved to + `deny=False, reason='no denied pattern matched'` even though real + bash genuinely runs `uv install foo` (confirmed live via a stand-in + `uv` binary on PATH: captured argv `uv called with: install foo`) -- + the DIRECT reference (`$TOOL $VERB foo`, no indirection) already + correctly denied via round 27's own fix; only the added `${!MREF}` + layer defeated it. Reproduced identically for `_rule_gh_api_write` + (`M=safe; M=$(echo POST); MREF=M; gh api repos/o/r/pulls/1/merge -X + ${!MREF}` -- real bash genuinely runs the write call with `-X POST`, + a genuine unreviewed write, e.g. merging a pull request) and for the + round-26 append/read/array-element classes threaded the identical + way. The sibling checkout/restore consumer (`_referenced_names`, + via `_resolve_path_tokens`) was NEVER affected -- it already does + the correct two-level resolution this function now also delegates + to, closing the asymmetry round 26 introduced when it added THIS + function as a narrower, parallel mechanism instead of reusing the + existing one.""" if not names: return False - for token in seg: - for match in _VAR_REF_FULL_RE.finditer(token): - braced_name, default_name, _default_text, indirect_name, unbraced_run = match.groups() - for name in (braced_name, default_name, indirect_name): - if name is not None and name in names: - return True - if unbraced_run is not None and any(unbraced_run[:i] in names for i in range(len(unbraced_run), 0, -1)): - return True - return False + return any(_referenced_names(token, name_to_raw_value, name_to_raw_value_history) & names for token in seg) _POISONED_REASSIGNMENT_GH_API_HIT = ( @@ -3513,6 +3536,7 @@ def _rule_gh_api_write( name_to_value: dict[str, str], name_to_raw_value: dict[str, str], names_poisoned_for_gh_api_and_b1: set[str] | None = None, + name_to_raw_value_history: dict[str, tuple[str, ...]] | None = None, ) -> str | None: """`literals` is already lowercased, matching the predecessor script's own case-insensitive match against its whole lowered command -- so @@ -3535,12 +3559,13 @@ def _rule_gh_api_write( already confirmed, rather than by deleting the name from NAME_TO_ VALUE/NAME_TO_RAW_VALUE.""" poisoned = names_poisoned_for_gh_api_and_b1 or set() + raw_history = name_to_raw_value_history or {} for seg in segments: literals = [t.lower() for t in seg if not _is_dynamic(t)] has_gh_api = any(literals[i : i + 2] == ["gh", "api"] for i in range(len(literals) - 1)) if not has_gh_api: continue - if _segment_references_a_name(seg, poisoned): + if _segment_references_a_name(seg, poisoned, name_to_raw_value, raw_history): return _POISONED_REASSIGNMENT_GH_API_HIT has_graphql = any(literals[i : i + 3] == ["gh", "api", "graphql"] for i in range(len(literals) - 2)) if has_graphql and "mutation" in lowered_command: @@ -5474,6 +5499,7 @@ def _segment_loop_hit( name_to_value: dict[str, str], name_to_raw_value: dict[str, str], names_poisoned_for_gh_api_and_b1: set[str] | None = None, + name_to_raw_value_history: dict[str, tuple[str, ...]] | None = None, ) -> tuple[str | None, bool]: """The B1a/B1b/B2/obfuscated-git-push-second-token loop -- factored out of `_classify_tokens` so it can be run TWICE: once against @@ -5518,8 +5544,9 @@ def _segment_loop_hit( every reference anywhere as a hit.""" is_git_push = False poisoned = names_poisoned_for_gh_api_and_b1 or set() + raw_history = name_to_raw_value_history or {} for seg in segments: - if seg and _is_dynamic(seg[0]) and _segment_references_a_name(seg, poisoned): + if seg and _is_dynamic(seg[0]) and _segment_references_a_name(seg, poisoned, name_to_raw_value, raw_history): return ( "a Bash command word is dynamically constructed from a variable that was reassigned " "after an earlier value in a way this classifier cannot soundly resolve (a compound " @@ -5802,22 +5829,34 @@ def _classify_tokens( return Verdict(True, literal_hit, is_git_push, checkout_restore_paths) gh_api_hit = _rule_gh_api_write( - segments, lowered_command, assigned, raw_assigned, names_poisoned_for_gh_api_and_b1 + segments, + lowered_command, + assigned, + raw_assigned, + names_poisoned_for_gh_api_and_b1, + raw_assigned_history, ) or _rule_gh_api_write( segments, lowered_command, assigned_write_biased, raw_assigned_write_biased, names_poisoned_for_gh_api_and_b1, + raw_assigned_history, ) if gh_api_hit: return Verdict(True, gh_api_hit, is_git_push, checkout_restore_paths) - loop_hit, loop_is_git_push = _segment_loop_hit(segments, assigned, raw_assigned, names_poisoned_for_gh_api_and_b1) + loop_hit, loop_is_git_push = _segment_loop_hit( + segments, assigned, raw_assigned, names_poisoned_for_gh_api_and_b1, raw_assigned_history + ) is_git_push = is_git_push or loop_is_git_push if not loop_hit: loop_hit, biased_loop_is_git_push = _segment_loop_hit( - segments, assigned_write_biased, raw_assigned_write_biased, names_poisoned_for_gh_api_and_b1 + segments, + assigned_write_biased, + raw_assigned_write_biased, + names_poisoned_for_gh_api_and_b1, + raw_assigned_history, ) is_git_push = is_git_push or biased_loop_is_git_push if loop_hit: @@ -5828,7 +5867,7 @@ def _classify_tokens( ] if collapsed_segments != segments: collapsed_hit, collapsed_is_git_push = _segment_loop_hit( - collapsed_segments, assigned, raw_assigned, names_poisoned_for_gh_api_and_b1 + collapsed_segments, assigned, raw_assigned, names_poisoned_for_gh_api_and_b1, raw_assigned_history ) is_git_push = is_git_push or collapsed_is_git_push if not collapsed_hit: @@ -5837,6 +5876,7 @@ def _classify_tokens( assigned_write_biased, raw_assigned_write_biased, names_poisoned_for_gh_api_and_b1, + raw_assigned_history, ) is_git_push = is_git_push or collapsed_biased_is_git_push if collapsed_hit: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 1a03a5c3..de9dbacf 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -351,6 +351,19 @@ def assert_allowed(command: str) -> None: "M=GET; M=$(echo x); M=HEAD; gh api repos/o/r/issues -X $M", "gh-api-method-value-reassigned-static-dynamic-static-stays-allowed", ), + # False-positive guard for the twenty-ninth-round `_segment_ + # references_a_name` indirect-reference fix (issue #1375): VERB is + # genuinely poisoned (static "harmless" then dynamic "install"), but + # the segment never references VERB itself -- only OTHER, indirectly, + # through MREF. The two-level resolution the fix now delegates to + # must not over-reach and treat every poisoned name anywhere in scope + # as reachable through an unrelated indirection. Confirmed live via a + # stand-in `uv` binary on PATH that this genuinely runs `uv status + # foo` -- "status" is not a watched verb. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); OTHER=status; MREF=OTHER; $TOOL ${!MREF} foo", + "indirect-ref-to-unrelated-name-stays-allowed-despite-a-poisoned-name-elsewhere", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1000,6 +1013,37 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "M=POST; M=$(echo x); M=GET; gh api repos/o/r/issues -X $M", "gh-api-method-value-ever-a-watched-write-method-stays-denied", ), + # Found live by Step 8 independent review, twenty-ninth round (issue + # #1375): `_segment_references_a_name`'s own round-26 single-level + # `${!NAME}` indirect-reference scan defeated EVERY poisoning class + # above (static-then-dynamic, append, read/array-element/printf-v) + # once the poisoned name was referenced through one extra layer of + # indirection -- the direct reference (no `${!MREF}`) already denied + # correctly; only wrapping it in an indirect reference bypassed + # detection. Confirmed live via a stand-in `uv` binary on PATH + # (captured argv: "install foo") that this genuinely runs `uv install + # foo`. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); MREF=VERB; $TOOL ${!MREF} foo", + "var-split-tool-and-verb-reassigned-from-static-referenced-indirectly", + ), + # Same round, the append counterpart referenced indirectly. + ( + "TOOL=uv; VERB=inst; VERB+=all; MREF=VERB; $TOOL ${!MREF} foo", + "var-split-tool-and-verb-reassigned-via-static-append-referenced-indirectly", + ), + # Same round, the `read` counterpart referenced indirectly. + ( + 'A=harmless; read A <<< "uv"; B=harmless2; read B <<< "install"; AREF=A; BREF=B; ${!AREF} ${!BREF} foo', + "var-split-tool-and-verb-reassigned-via-read-referenced-indirectly", + ), + # Same round, the gh-api-write counterpart: real bash genuinely ran + # `gh api repos/o/r/pulls/1/merge -X POST` (captured argv confirms + # it), a genuine, unreviewed write API call. + ( + "M=safe; M=$(echo POST); MREF=M; gh api repos/o/r/pulls/1/merge -X ${!MREF}", + "gh-api-method-value-reassigned-from-a-static-value-referenced-indirectly", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index b4bdfe91..021865f0 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2342,24 +2342,71 @@ def test_names_reassigned_by_read_or_printf_matches_model_for_a_printf_v_target( def test_segment_references_a_name_finds_a_bare_reference() -> None: """`_segment_references_a_name` recognizes a plain `$NAME` reference to a poisoned name.""" - assert checker._segment_references_a_name(["gh", "api", "repos/x/y", "-X", "$M"], {"M"}) is True + assert checker._segment_references_a_name(["gh", "api", "repos/x/y", "-X", "$M"], {"M"}, {}, {}) is True def test_segment_references_a_name_finds_a_braced_reference() -> None: - assert checker._segment_references_a_name(["echo", "${M}"], {"M"}) is True + assert checker._segment_references_a_name(["echo", "${M}"], {"M"}, {}, {}) is True def test_segment_references_a_name_ignores_an_unrelated_reference() -> None: """No false positive: a reference to a name that is NOT in the poisoned set is ignored.""" - assert checker._segment_references_a_name(["echo", "$OTHER"], {"M"}) is False + assert checker._segment_references_a_name(["echo", "$OTHER"], {"M"}, {}, {}) is False def test_segment_references_a_name_empty_names_is_always_false() -> None: """No poisoned names at all -- the common case for every pre-existing call site -- never reports a reference, regardless of segment content.""" - assert checker._segment_references_a_name(["gh", "api", "repos/x/y", "-X", "$M"], set()) is False + assert checker._segment_references_a_name(["gh", "api", "repos/x/y", "-X", "$M"], set(), {}, {}) is False + + +def test_segment_references_a_name_finds_an_indirect_reference() -> None: + """Round 29 (issue #1375): a poisoned name referenced through one + extra `${!MREF}` indirection layer is still found -- `_segment_ + references_a_name` now delegates to `_referenced_names`'s own + two-level resolution instead of the round-26 single-level scan that + missed this. MREF's raw value is "M" (the poisoned name itself), so + `${!MREF}` resolves through MREF to M.""" + assert ( + checker._segment_references_a_name( + ["gh", "api", "repos/x/y", "-X", "${!MREF}"], + {"M"}, + {"MREF": "M"}, + {}, + ) + is True + ) + + +def test_segment_references_a_name_ignores_an_unrelated_indirect_reference() -> None: + """No false positive: an indirect reference through a name whose + value does NOT point at a poisoned name is ignored.""" + assert ( + checker._segment_references_a_name( + ["gh", "api", "repos/x/y", "-X", "${!MREF}"], + {"M"}, + {"MREF": "OTHER"}, + {}, + ) + is False + ) + + +@_PROPERTIES +@given(name=_IDENTIFIERS, ref_name=_IDENTIFIERS) +def test_segment_references_a_name_matches_model_for_any_indirect_reference(name: str, ref_name: str) -> None: + """Model-based: for ANY poisoned NAME and any distinct REF_NAME whose + own raw value names it, a segment referencing NAME only through + `${!REF_NAME}` is still found -- the round-29 fix's own two-level + delegation to `_referenced_names` must hold for every identifier + shape `_IDENTIFIERS` generates, not just the specific example pinned + above. A REF_NAME colliding with NAME itself degenerates to the + already-covered direct-reference case, so it is excluded here.""" + assume(ref_name != name) + seg = ["gh", "api", "repos/x/y", "-X", f"${{!{ref_name}}}"] + assert checker._segment_references_a_name(seg, {name}, {ref_name: name}, {}) is True def test_rule_gh_api_write_denies_a_reference_to_a_poisoned_name() -> None: @@ -2717,6 +2764,43 @@ def test_classify_still_denies_a_gh_api_method_that_was_ever_a_watched_write_met assert verdict.deny is True +def test_classify_denies_a_static_then_dynamic_tool_and_verb_referenced_indirectly() -> None: + """CRITICAL bypass regression pin (round-29 independent review, issue + #1375): `_segment_references_a_name`'s own round-26 single-level + `${!NAME}` indirect-reference scan missed a poisoned name referenced + through one extra layer of indirection -- the direct reference + (round 27's own pin, `test_classify_denies_a_var_split_tool_and_verb_ + reassigned_from_a_static_value` or equivalent) already denied + correctly; only wrapping it in `${!MREF}` bypassed detection. + Confirmed live via a stand-in `uv` binary on PATH that this genuinely + runs `uv install foo`.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); MREF=VERB; $TOOL ${!MREF} foo") + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_referenced_indirectly() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write` (round 29, + issue #1375). Confirmed live via a stand-in `gh` binary on PATH: this + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST`, a genuine, + unreviewed write API call.""" + verdict = checker.classify("M=safe; M=$(echo POST); MREF=M; gh api repos/o/r/pulls/1/merge -X ${!MREF}") + assert verdict.deny is True + + +def test_classify_allows_an_indirect_reference_to_an_unrelated_name_despite_a_poisoned_name_elsewhere() -> None: + """No over-denial: an indirect `${!MREF}` reference resolving to a + name that was never poisoned must stay allowed, even though a + DIFFERENT, genuinely poisoned name exists elsewhere in scope -- the + round-29 fix's two-level resolution must not treat every poisoned + name anywhere as reachable through an unrelated indirection. + Confirmed live via a stand-in `uv` binary on PATH that this genuinely + runs `uv status foo` -- "status" is not a watched verb.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); OTHER=status; MREF=OTHER; $TOOL ${!MREF} foo" + ) + assert verdict.deny is False + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 0794ec6ca4171f01dccd1f92d1179bee4eab8c44 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 01:10:21 +0000 Subject: [PATCH 35/46] fix(hooks): clear gh-api-write/B1a-B1b poisoning after any later static reassignment A fresh, independent adversarial review of this PR's current head (round 30) found that round 28's own "clear on later static reassignment" fix was applied only inside `_names_reassigned_from_a_ static_value` -- the OTHER two components of `_names_poisoned_for_gh_ api_and_b1`'s own union, `_names_appended_to` and `_names_reassigned_ by_untracked_construct` (compound `+=` append, array-element assignment, and `read`/`readarray`/`mapfile`/`printf -v` reassignment), still only ever ADD to their own result and never clear it -- so a name reassigned via one of those constructs and THEN given an ordinary, later static value that fully restores trust stayed poisoned forever, the identical false-positive over-denial shape round 28 already fixed, just left open for these two other classes. Independently reproduced live before acting, via `classify()` and real bash execution with stand-in `uv`/`gh` binaries on PATH: `TOOL=uv; VERB=inst; VERB+=all; VERB=safe; $TOOL $VERB foo` genuinely runs `uv safe foo` (`safe` is not a watched verb; captured argv: "safe foo") but was wrongly denied by B1a/B1b. The `read` and array-element counterparts (`TOOL=uv; read VERB <<< status; VERB=safe; $TOOL $VERB foo` and `TOOL=uv; VERB[0]=install; VERB=safe; $TOOL $VERB foo`) reproduce identically. `M=P; M+=OST; M=GET; gh api repos/o/r/issues -X $M` reproduces the same shape for `_rule_gh_api_write` (real bash genuinely runs `-X GET`, a read method). Confirmed through the real wrapper: all four now allow with exit 0 where they previously denied with exit 2. Closed by a new `_names_cleared_by_a_later_static_reassignment` function: for each candidate name, it determines whether that name's LAST reassignment-class event anywhere in the command -- append, array-element assignment, read/printf-v reassignment, or a plain `NAME=value` assignment (static or dynamic) -- is a plain, non-dynamic assignment, in true token/segment order across all four event types combined. `_names_poisoned_for_gh_api_and_b1` now subtracts this cleared set from its own raw union as a final step, generalizing round 28's "trust only the name's latest assignment" principle across all three of its component classes instead of only the one it originally covered. Deliberately NOT applied to `_names_appended_to`/`_names_ reassigned_by_untracked_construct` themselves, since checkout/restore's own `_names_with_dynamic_assignment` also consumes them and its own "poison outright, never clear" posture for these classes was independently confirmed correct and deliberate during round 27's own review -- this fix only post-processes the combined set gh-api-write/ B1a-B1b actually use, leaving checkout/restore's own behavior unchanged. A no-under-correction control was also verified and pinned: a name given a static value and THEN appended to (`TOOL=uv; VERB=safe; VERB+=x; $TOOL $VERB foo`) correctly stays denied, since the append -- not the earlier static value -- is the name's latest event and its own combined value is still unrecoverable in general, exactly as round 25's own original "poison unconditionally on any append" posture already requires. Regression tests added at every established layer: unit tests for `_names_cleared_by_a_later_static_reassignment` (append/read/array- element/printf-v each cleared by a later static value; a static-then- append no-under-correction case; false-positive guards for an unrelated name via each event type), a `@given` Hypothesis property test exercising it directly, integration-level unit tests confirming `_names_poisoned_for_gh_api_and_b1` itself reflects the clearing, `classify()`-level end-to-end pins for every reported shape plus the no-under-correction control, and wrapper-level end-to-end pins against a real stand-in `uv`/`gh` binary (hooks/test_gitapex_check_bash_ safety.py, tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI's own whole- codebase invocation), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-30 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 102 ++++++++++- hooks/test_gitapex_check_bash_safety.py | 45 +++++ ...st_gitapex_check_bash_safety_properties.py | 168 ++++++++++++++++++ 3 files changed, 313 insertions(+), 2 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index ab90ec76..9e4ec4ea 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4181,6 +4181,94 @@ def _names_reassigned_from_a_static_value(tokens: list[str]) -> set[str]: return poisoned +def _names_cleared_by_a_later_static_reassignment(tokens: list[str], candidates: set[str]) -> set[str]: + """Every NAME in CANDIDATES whose LAST reassignment-class event + anywhere in TOKENS -- a compound `+=` append, an array-element + assignment, a `read`/`readarray`/`mapfile`/`printf -v` reassignment, + or a plain `NAME=value` assignment (static or dynamic) -- is a + plain, non-dynamic assignment, strictly after every other kind of + event for that name. A later static assignment fully overwrites and + restores a known, trustworthy value -- the genuine value real bash + uses at the point of use -- regardless of what unresolvable state + came before it: the same "trust only the name's LATEST assignment" + principle round 28 already established for `_names_reassigned_ + from_a_static_value`'s own narrower static-then-dynamic class, + generalized here across ALL of `_names_poisoned_for_gh_api_and_b1`'s + own component classes. + + CRITICAL bug found by independent adversarial review (round 30, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: round 28's own "clear on later static reassignment" + fix was applied ONLY inside `_names_reassigned_from_a_static_value` + -- `_names_appended_to` and `_names_reassigned_by_untracked_ + construct` (the OTHER two components of `_names_poisoned_for_gh_ + api_and_b1`'s own union) still only ever ADD to their own result, + never clear it, so a name reassigned via append/`read`/an array- + element assignment and THEN later given an ordinary, fully- + trustworthy static value stayed poisoned forever -- the identical + false-positive over-denial shape round 28 already fixed, just left + open for these two other classes. `TOOL=uv; VERB=inst; VERB+=all; + VERB=safe; $TOOL $VERB foo` -- real bash's final value of `$VERB` is + `safe`, not a watched verb at all (confirmed live via a stand-in + `uv` binary on PATH: captured argv `uv called with: safe foo`) -- + was wrongly denied by B1a/B1b before this fix. `M=P; M+=OST; M=GET; + gh api repos/o/r/issues -X $M` reproduces the identical shape for + `_rule_gh_api_write` (GET is a read method, confirmed live via a + stand-in `gh` binary on PATH: captured argv `gh called with: api + repos/o/r/issues -X GET`). The `read`- and array-element-reassigned + counterparts (`TOOL=uv; read VERB <<< status; VERB=safe; $TOOL + $VERB foo` and `TOOL=uv; VERB[0]=install; VERB=safe; $TOOL $VERB + foo`) reproduce identically (captured argv: `uv called with: safe + foo` for both). + + Deliberately NOT applied to `_names_appended_to`/`_names_ + reassigned_by_untracked_construct` themselves -- those two functions + are also shared by `_names_with_dynamic_assignment` (checkout/ + restore's own full union), whose own "poison outright, never clear" + posture for these classes was independently confirmed correct and + deliberate during round 27's own review (see that function's own + docstring); this function instead post-processes only the poisoned + set `_names_poisoned_for_gh_api_and_b1` itself returns, leaving + those two shared functions -- and checkout/restore's own + consumption of them -- completely unchanged.""" + if not candidates: + return set() + last_is_static: dict[str, bool] = {} + for seg in segment_tokens(tokens): + if seg and not _is_dynamic(seg[0]): + head = seg[0].lower() + if head in _READ_COMMAND_WORDS: + for tok in seg[1:]: + if not _is_dynamic(tok) and tok in candidates and _BARE_IDENTIFIER_RE.match(tok): + last_is_static[tok] = False + elif head == "printf": + for i, tok in enumerate(seg): + if tok == "-v" and i + 1 < len(seg): + target = seg[i + 1] + if not _is_dynamic(target) and target in candidates and _BARE_IDENTIFIER_RE.match(target): + last_is_static[target] = False + for token in seg: + array_match = _ARRAY_ELEMENT_ASSIGN_RE.match(token) + if array_match: + name = array_match.group(1) + if name in candidates: + last_is_static[name] = False + continue + append_match = _APPEND_ASSIGN_RE.match(token) + if append_match: + name = append_match.group(1) + if name in candidates: + last_is_static[name] = False + continue + match = _ASSIGN_RE.match(token) + if match: + name = match.group(1) + if name in candidates: + last_is_static[name] = not _is_dynamic(token) + return {name for name, is_static in last_is_static.items() if is_static} + + def _names_poisoned_for_gh_api_and_b1(tokens: list[str]) -> set[str]: """The full poisoned-names set `_rule_gh_api_write` and `_segment_ loop_hit` (B1a/B1b) actually consume -- the union of every class of @@ -4208,12 +4296,22 @@ def _names_poisoned_for_gh_api_and_b1(tokens: list[str]) -> set[str]: candidate list reads as "no evidence of a write," the deliberately- accepted posture for that specific, disclosed gap) -- poisoning it here would regress that case into a false-positive deny, exactly - what round 26's own (too-broad) exclusion was trying to prevent.""" - return ( + what round 26's own (too-broad) exclusion was trying to prevent. + + Round 30 (issue #1375): the raw union above still only ever ADDS a + name -- `_names_reassigned_from_a_static_value`'s own internal + clearing only covers ITS OWN static-then-dynamic class, not the + append/untracked-construct classes. `_names_cleared_by_a_later_ + static_reassignment` (see that function's own docstring for the + live bypass this restores) is applied as a final subtraction so the + combined result genuinely tracks each name's LATEST assignment-class + event, whichever of the three classes it came from.""" + poisoned = ( _names_appended_to(tokens) | _names_reassigned_from_a_static_value(tokens) | _names_reassigned_by_untracked_construct(tokens) ) + return poisoned - _names_cleared_by_a_later_static_reassignment(tokens, poisoned) def _names_reassigned_by_read_or_printf(tokens: list[str]) -> set[str]: diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index de9dbacf..b622c7c7 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -364,6 +364,34 @@ def assert_allowed(command: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); OTHER=status; MREF=OTHER; $TOOL ${!MREF} foo", "indirect-ref-to-unrelated-name-stays-allowed-despite-a-poisoned-name-elsewhere", ), + # False-positive guard for the thirtieth-round `_names_cleared_by_a_ + # later_static_reassignment` fix (issue #1375): round 28's own + # "clear on later static reassignment" fix was applied only inside + # `_names_reassigned_from_a_static_value`, leaving an append/`read`/ + # array-element reassignment poisoned forever even after a later, + # fully-trustworthy static value. Confirmed live via a stand-in `uv` + # binary on PATH that this genuinely runs `uv safe foo` -- "safe" is + # not a watched verb. + ( + "TOOL=uv; VERB=inst; VERB+=all; VERB=safe; $TOOL $VERB foo", + "var-split-tool-and-verb-appended-then-given-a-later-static-value-stays-allowed", + ), + # Same round, the `read` counterpart. + ( + "TOOL=uv; read VERB <<< status; VERB=safe; $TOOL $VERB foo", + "var-split-tool-and-verb-read-into-then-given-a-later-static-value-stays-allowed", + ), + # Same round, the array-element-assignment counterpart. + ( + "TOOL=uv; VERB[0]=install; VERB=safe; $TOOL $VERB foo", + "var-split-tool-and-verb-array-element-assigned-then-given-a-later-static-value-stays-allowed", + ), + # Same round, the gh-api-write counterpart: real bash genuinely runs + # `gh api repos/o/r/issues -X GET`, a read method. + ( + "M=P; M+=OST; M=GET; gh api repos/o/r/issues -X $M", + "gh-api-method-value-appended-then-given-a-later-static-value-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1044,6 +1072,23 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "M=safe; M=$(echo POST); MREF=M; gh api repos/o/r/pulls/1/merge -X ${!MREF}", "gh-api-method-value-reassigned-from-a-static-value-referenced-indirectly", ), + # Found live by Step 8 independent review, thirtieth round (issue + # #1375): the round-30 no-under-correction guard -- a name given a + # static value and THEN appended to must stay denied, since the + # append (not the earlier static value) is the name's latest + # assignment-class event and its own combined value is still + # unrecoverable in general, exactly as round 25's own original + # "poison unconditionally on any append" posture already requires. + # Confirmed live via a stand-in `uv` binary on PATH that this + # particular instance happens to resolve to the harmless `uv safex + # foo` -- the classifier's own deliberately conservative posture + # correctly denies it anyway, since it cannot in general predict a + # concatenation's own final value from a static prefix and a + # dynamic append alone. + ( + "TOOL=uv; VERB=safe; VERB+=x; $TOOL $VERB foo", + "var-split-tool-and-verb-given-a-static-value-then-appended-to-stays-denied", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 021865f0..2f80ecd6 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2679,6 +2679,127 @@ def test_names_poisoned_for_gh_api_and_b1_excludes_a_dynamic_only_name() -> None assert checker._names_poisoned_for_gh_api_and_b1(["Q=$A$B"]) == set() +def test_names_cleared_by_a_later_static_reassignment_empty_candidates_is_always_empty() -> None: + """No candidates, no work: the common case for a command with no + poisoned names at all never scans TOKENS.""" + assert checker._names_cleared_by_a_later_static_reassignment(["VERB=safe"], set()) == set() + + +def test_names_cleared_by_a_later_static_reassignment_clears_after_an_append() -> None: + """Regression pin for the real false-positive over-denial found live + by Step 8 independent review, thirtieth round (issue #1375): a name + appended to and THEN given a later, ordinary static value must be + cleared -- the append is no longer the name's latest assignment- + class event.""" + tokens = ["VERB=inst", "VERB+=all", "VERB=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + +def test_names_cleared_by_a_later_static_reassignment_clears_after_a_read() -> None: + tokens = ["read", "VERB", "VERB=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + +def test_names_cleared_by_a_later_static_reassignment_clears_after_an_array_element_assignment() -> None: + tokens = ["VERB[0]=install", "VERB=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_when_the_append_is_last() -> None: + """No under-correction: a name given a static value and THEN + appended to must NOT be cleared -- the append is the name's latest + assignment-class event, so its own combined value is still + unrecoverable.""" + tokens = ["VERB=safe", "VERB+=x"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_ignores_an_unrelated_name() -> None: + """No false positive: a static reassignment to a name NOT in + CANDIDATES must not spuriously clear anything.""" + tokens = ["OTHER+=x", "OTHER=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_clears_after_a_printf_v() -> None: + tokens = ["printf", "-v", "VERB", "%s", "install", ";", "VERB=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + +def test_names_cleared_by_a_later_static_reassignment_ignores_an_unrelated_printf_v_target() -> None: + """No false positive: a `printf -v` target NOT in CANDIDATES must + not spuriously mark it for clearing.""" + tokens = ["printf", "-v", "OTHER", "%s", "x", ";", "OTHER=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_ignores_an_unrelated_array_element_assignment() -> None: + """No false positive: an array-element assignment to a name NOT in + CANDIDATES must not spuriously mark it for clearing.""" + tokens = ["OTHER[0]=x", "OTHER=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +@_PROPERTIES +@given(name=_IDENTIFIERS, appended_value=_VALUES, final_static_value=_VALUES) +def test_names_cleared_by_a_later_static_reassignment_matches_model_for_an_append_then_a_later_static_value( + name: str, appended_value: str, final_static_value: str +) -> None: + """Model-based, exercising `_names_cleared_by_a_later_static_ + reassignment` directly (issue #1178's own detection-logic property- + coverage requirement): for ANY identifier appended to and then given + a later static value, the function always includes it in the + cleared set -- and a second, entirely unrelated identifier whose OWN + latest event is an append with no later static value is never + included alongside it, even though it is a member of CANDIDATES + too.""" + other_name = name + "_OTHER" + tokens = [ + f"{name}=x", + f"{name}+={appended_value}", + f"{name}={final_static_value}", + f"{other_name}=x", + f"{other_name}+={appended_value}", + ] + result = checker._names_cleared_by_a_later_static_reassignment(tokens, {name, other_name}) + assert name in result + assert other_name not in result + + +@_PROPERTIES +@given(name=_IDENTIFIERS, appended_value=_VALUES, final_static_value=_VALUES) +def test_names_poisoned_for_gh_api_and_b1_matches_model_for_an_append_then_a_later_static_value( + name: str, appended_value: str, final_static_value: str +) -> None: + """Model-based: for ANY identifier appended to and then given a + later static value, `_names_poisoned_for_gh_api_and_b1` never + includes it -- the round-30 fix's own generalized "trust only the + name's latest assignment" principle must hold for every identifier + shape `_IDENTIFIERS` generates, not just the specific example pinned + above.""" + tokens = [f"{name}=x", f"{name}+={appended_value}", f"{name}={final_static_value}"] + assert name not in checker._names_poisoned_for_gh_api_and_b1(tokens) + + +def test_names_poisoned_for_gh_api_and_b1_clears_an_append_given_a_later_static_value() -> None: + """Integration-level pin: the round-30 fix's subtraction actually + reaches `_names_poisoned_for_gh_api_and_b1`'s own combined result, + not just the standalone `_names_cleared_by_a_later_static_ + reassignment` helper.""" + tokens = ["VERB=inst", "VERB+=all", "VERB=safe"] + assert checker._names_poisoned_for_gh_api_and_b1(tokens) == set() + + +def test_names_poisoned_for_gh_api_and_b1_clears_a_read_reassignment_given_a_later_static_value() -> None: + tokens = ["read", "VERB", "VERB=safe"] + assert checker._names_poisoned_for_gh_api_and_b1(tokens) == set() + + +def test_names_poisoned_for_gh_api_and_b1_clears_an_array_element_assignment_given_a_later_static_value() -> None: + tokens = ["VERB[0]=install", "VERB=safe"] + assert checker._names_poisoned_for_gh_api_and_b1(tokens) == set() + + def test_classify_denies_a_b1b_tool_and_verb_reassigned_via_plain_dynamic_reassignment() -> None: """End-to-end regression pin for the round-27 finding at the `classify()` level, against B1b. Confirmed live before this fix via @@ -2801,6 +2922,53 @@ def test_classify_allows_an_indirect_reference_to_an_unrelated_name_despite_a_po assert verdict.deny is False +def test_classify_allows_a_b1b_tool_and_verb_appended_then_given_a_later_static_value() -> None: + """CRITICAL false-positive regression pin (round-30 independent + review, issue #1375): round 28's own "clear on later static + reassignment" fix was applied only inside `_names_reassigned_from_ + a_static_value`, leaving `_names_appended_to`/`_names_reassigned_by_ + untracked_construct` (the OTHER two components of `_names_poisoned_ + for_gh_api_and_b1`'s own union) poisoning a name forever once + appended to, even after a later static value fully restores trust. + Confirmed live via a stand-in `uv` binary on PATH that this + genuinely runs `uv safe foo` -- "safe" is not a watched verb.""" + verdict = checker.classify("TOOL=uv; VERB=inst; VERB+=all; VERB=safe; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_gh_api_method_appended_then_given_a_later_static_value() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`. + Confirmed live via a stand-in `gh` binary on PATH that this + genuinely runs `gh api repos/o/r/issues -X GET`, a read method.""" + verdict = checker.classify("M=P; M+=OST; M=GET; gh api repos/o/r/issues -X $M") + assert verdict.deny is False + + +def test_classify_allows_a_b1b_tool_and_verb_read_into_then_given_a_later_static_value() -> None: + """Same round, the `read` counterpart. Confirmed live via a stand-in + `uv` binary on PATH that this genuinely runs `uv safe foo`.""" + verdict = checker.classify("TOOL=uv; read VERB <<< status; VERB=safe; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_b1b_tool_and_verb_array_element_assigned_then_given_a_later_static_value() -> None: + """Same round, the array-element-assignment counterpart. Confirmed + live via a stand-in `uv` binary on PATH that this genuinely runs + `uv safe foo`.""" + verdict = checker.classify("TOOL=uv; VERB[0]=install; VERB=safe; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_still_denies_a_b1b_tool_and_verb_given_a_static_value_then_appended_to() -> None: + """No under-correction: a name given a static value and THEN + appended to must stay denied -- the append is the name's latest + assignment-class event, so its own combined value is still + unrecoverable, regardless of how harmless the earlier static value + looked.""" + verdict = checker.classify("TOOL=uv; VERB=safe; VERB+=x; $TOOL $VERB foo") + assert verdict.deny is True + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 86751c9446b8e4139acc1c964cdfbfd20c4c3b2e Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 01:29:00 +0000 Subject: [PATCH 36/46] fix(hooks): never let a subshell-scoped static reassignment clear gh-api-write/B1a-B1b poisoning A fresh, independent adversarial review of this PR's current head (round 31) found the most severe defect in this file's own review history to date: a genuine security bypass (under-denial), not merely a false-positive over-denial like rounds 28/30. `segment_tokens` treats `(`/`)` as pure sequencing boundaries with no subshell-scope tracking at all -- but real bash's own `(...)` grouping runs in a forked, isolated shell whose own assignments never propagate back to the parent. Both `_names_reassigned_from_a_static_value` (round 27/28's own "clear on later static reassignment" logic) and round 30's own `_names_cleared_by_a_later_static_reassignment` treated a static assignment written INSIDE `(...)` exactly like an ordinary top-level one, letting an attacker (or an unwitting agent) wrap a fake "the value is safe now" reassignment in parens to escape a genuine poisoning that a prior append/read/array-element/dynamic reassignment had already established. Independently reproduced live before acting, via `classify()` and real bash execution with stand-in `uv`/`gh` binaries on PATH, across every poisoning class this file tracks: `TOOL=uv; VERB=harmless; VERB=$(echo install); (VERB=safe); $TOOL $VERB foo` resolved to `deny=False` even though real bash genuinely runs `uv install foo`, NOT `safe foo` -- the parenthesized `VERB=safe` never reaches the parent shell's own `$VERB` at all (confirmed live via a stand-in `uv` binary on PATH: captured argv `uv called with: install foo`). The append, array-element, and `read` counterparts, plus the `gh api` write-method counterpart (`M=safe; M=$(echo POST); (M=GET); gh api repos/o/r/pulls/1/merge -X $M`, real bash genuinely running `-X POST`, a genuine unreviewed pull-request merge), all reproduce identically. Confirmed through the real wrapper: all five now deny with exit 2 where they previously allowed with exit 0. Checkout/restore's own `_names_with_dynamic_assignment` was independently confirmed unaffected (it never clears at all, regardless of depth, so it already denied this shape correctly). Closed by two new helpers -- `_paren_depths` (per-token subshell nesting depth over a flat token list) and `_segment_tokens_with_subshell_depth` (the same, paired with `segment_tokens`'s own per-segment output, since a single segment's own depth is always internally consistent) -- and threading depth awareness through both vulnerable functions: a plain static assignment may only clear/restore trust when it occurs at depth 0 (true top-level scope); one found at depth 1+ is treated as invisible to the parent scope, leaving whatever state a prior event already recorded untouched. A dynamic reassignment, append, array-element assignment, or `read`/`printf -v` reassignment still poisons unconditionally regardless of depth, since staying conservative about what a subshell might have done is always the safe direction, never the dangerous one. An earlier, harmless subshell no longer blocks a genuine LATER top-level static reassignment from clearing normally, confirmed with dedicated no-over-correction regression tests and controls. Regression tests added at every established layer: unit tests for `_paren_depths`/`_segment_tokens_with_subshell_depth` directly, unit and `@given` Hypothesis property tests for both fixed functions' subshell-blindness closure and no-over-correction behavior, integration- level `classify()` end-to-end pins for every reported bypass shape plus two false-positive controls, and wrapper-level end-to-end pins against a real stand-in `uv`/`gh` binary (hooks/test_gitapex_check_bash_ safety.py, tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format (including two S105/B905 findings ruff itself flagged -- a loop variable literally named `token` compared against a string literal false-positives bandit's hardcoded-credential heuristic, renamed to the file's own established `tok` convention; an explicit `zip(..., strict=True)`, matching this file's own existing convention), mypy, xenon (CI's own whole-codebase invocation), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-31 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 142 ++++++++++++++++-- hooks/test_gitapex_check_bash_safety.py | 52 +++++++ ...st_gitapex_check_bash_safety_properties.py | 142 ++++++++++++++++++ 3 files changed, 327 insertions(+), 9 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 9e4ec4ea..f35c7dd7 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4073,6 +4073,63 @@ def _names_reassigned_by_untracked_construct(tokens: list[str]) -> set[str]: return names +def _paren_depths(tokens: list[str]) -> list[int]: + """The `(...)`-subshell nesting depth of each token in TOKENS, as a + list the same length and order as TOKENS itself (0 = top-level + scope, 1+ = inside one or more subshell groupings). `(`/`)` + themselves each get the depth in effect AT that boundary token (the + new, deeper depth for `(`; the old, shallower depth for `)`) -- + irrelevant in practice since a caller only ever consults the depth + of an actual assignment-shaped content token, never a bare `(`/`)`. + + Added by round 31 (issue #1375): `segment_tokens` treats `(`/`)` + as pure sequencing boundaries with no depth tracking at all -- + correct for simple-command splitting, but insufficient for a caller + that needs to know whether a given assignment can actually affect a + reference OUTSIDE its own subshell scope. Real bash's own subshell + semantics: a `(...)` grouping runs in a forked, isolated shell whose + own assignments never propagate back to the parent -- see + `_names_reassigned_from_a_static_value`'s own docstring for the live + bypass this closes.""" + depths: list[int] = [] + depth = 0 + for tok in tokens: + if tok == "(": + depth += 1 + depths.append(depth) + elif tok == ")": + depths.append(depth) + depth = max(depth - 1, 0) + else: + depths.append(depth) + return depths + + +def _segment_tokens_with_subshell_depth(tokens: list[str]) -> list[tuple[list[str], int]]: + """Like `segment_tokens`, but pairs each returned segment with its + own `(...)`-subshell nesting DEPTH (see `_paren_depths`'s own + docstring) at the point it appears in TOKENS. A single segment's own + depth is always internally consistent -- `(`/`)` are pure segment + boundaries in `segment_tokens` too, so no segment can itself + straddle a depth change -- added by round 31 (issue #1375) for + `_names_cleared_by_a_later_static_reassignment`'s own use; see that + function's own docstring for the live bypass this closes.""" + segments: list[tuple[list[str], int]] = [([], 0)] + depth = 0 + for tok in tokens: + if tok == "(": + depth += 1 + segments.append(([], depth)) + elif tok == ")": + depth = max(depth - 1, 0) + segments.append(([], depth)) + elif tok in _SINGLE_OPS or tok in _MULTI_OPS: + segments.append(([], depth)) + else: + segments[-1][0].append(tok) + return [(seg, d) for seg, d in segments if seg] + + def _names_reassigned_from_a_static_value(tokens: list[str]) -> set[str]: """Every NAME that carried a trustworthy STATIC value at some point in TOKENS and was LATER reassigned a DYNAMIC (`$`/backtick- @@ -4164,18 +4221,51 @@ def _names_reassigned_from_a_static_value(tokens: list[str]) -> set[str]: reassignment after this one still correctly re-poisons the name -- verified live: `M=GET; M=$(echo x); M=HEAD; M=$(echo other)` leaves `M` poisoned again, matching real bash's own final, genuinely- - dynamic value).""" + dynamic value). + + CRITICAL bypass found by independent adversarial review (round 31, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: round 28's own "clear on later static reassignment" + fix above did not account for bash's own subshell scoping at all -- + a `(...)` grouping runs in a forked, isolated shell whose own + assignments never propagate back to the parent, but this function's + own flat, depth-blind scan over TOKENS treated a static assignment + written INSIDE `(...)` exactly like an ordinary top-level one, + letting it wrongly clear a genuinely poisoned name. `TOOL=uv; + VERB=harmless; VERB=$(echo install); (VERB=safe); $TOOL $VERB foo` + resolved to `deny=False, reason='no denied pattern matched'` even + though real bash genuinely runs `uv install foo` -- the parenthesized + `VERB=safe` never actually reaches the parent shell's own `$VERB` at + all (confirmed live via a stand-in `uv` binary on PATH: captured + argv `uv called with: install foo`, NOT `safe foo`). Reproduced + identically for `_rule_gh_api_write`: `M=safe; M=$(echo POST); + (M=GET); gh api repos/o/r/pulls/1/merge -X $M` also resolved to + `deny=False` -- real bash genuinely runs `-X POST` (captured argv + confirms it), a genuine unreviewed write. Closed by tracking each + assignment token's own `(...)`-nesting depth via `_paren_depths` and + only letting a static reassignment clear POISONED when it occurs at + depth 0 (true top-level scope, the only scope whose own reassignment + can actually affect a top-level reference); a static assignment + found at depth 1+ is treated as invisible to the parent scope -- + neither clearing nor registering EVER_STATIC -- while a DYNAMIC + reassignment at ANY depth still poisons unconditionally, since + staying conservative about what a subshell might have done is always + the safe direction, never the dangerous one. Checkout/restore's own + `_names_with_dynamic_assignment` was independently confirmed + unaffected by this exact bypass shape (it never clears at all, + regardless of depth, so it already denied this shape correctly).""" ever_static: set[str] = set() poisoned: set[str] = set() - for token in tokens: - match = _ASSIGN_RE.match(token) + for tok, depth in zip(tokens, _paren_depths(tokens), strict=True): + match = _ASSIGN_RE.match(tok) if not match: continue name = match.group(1) - if _is_dynamic(token): + if _is_dynamic(tok): if name in ever_static: poisoned.add(name) - else: + elif depth == 0: ever_static.add(name) poisoned.discard(name) return poisoned @@ -4231,11 +4321,38 @@ def _names_cleared_by_a_later_static_reassignment(tokens: list[str], candidates: docstring); this function instead post-processes only the poisoned set `_names_poisoned_for_gh_api_and_b1` itself returns, leaving those two shared functions -- and checkout/restore's own - consumption of them -- completely unchanged.""" + consumption of them -- completely unchanged. + + CRITICAL bypass found by independent adversarial review (round 31, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: this function's own original form was, like + `_names_reassigned_from_a_static_value`'s own pre-round-31 form, + completely blind to bash's own subshell scoping -- a static + assignment written INSIDE a `(...)` grouping never actually reaches + the parent shell's own copy of the name, but the original per- + segment scan treated it exactly like an ordinary top-level clearing + assignment. `TOOL=uv; VERB=inst; VERB+=all; (VERB=safe); $TOOL + $VERB foo` resolved to `deny=False` even though real bash genuinely + runs `uv install foo`, NOT `safe foo` (confirmed live via a stand-in + `uv` binary on PATH: captured argv `uv called with: install foo`); + the `read`- and array-element-reassigned counterparts reproduce + identically. Closed the same way as `_names_reassigned_from_a_ + static_value`'s own round-31 fix: each segment's own `(...)`-nesting + depth (via `_segment_tokens_with_subshell_depth`) gates whether a + plain static assignment in that segment may set a candidate's own + LAST-event state to "cleared" -- only a depth-0 (true top-level) + static assignment may do so; a depth-1+ one is treated as invisible + to the parent scope, leaving whatever state a prior event already + recorded untouched. Every OTHER event class (append, array-element, + read/printf-v, and a dynamic plain assignment) still marks the + candidate "not cleared" regardless of depth, since staying + conservative about what a subshell might have done is always the + safe direction, never the dangerous one.""" if not candidates: return set() last_is_static: dict[str, bool] = {} - for seg in segment_tokens(tokens): + for seg, depth in _segment_tokens_with_subshell_depth(tokens): if seg and not _is_dynamic(seg[0]): head = seg[0].lower() if head in _READ_COMMAND_WORDS: @@ -4264,8 +4381,15 @@ def _names_cleared_by_a_later_static_reassignment(tokens: list[str], candidates: match = _ASSIGN_RE.match(token) if match: name = match.group(1) - if name in candidates: - last_is_static[name] = not _is_dynamic(token) + if name not in candidates: + continue + if _is_dynamic(token): + last_is_static[name] = False + elif depth == 0: + last_is_static[name] = True + # depth > 0 and static: a subshell-scoped reassignment + # never reaches the parent scope -- leave any existing + # state untouched (round 31). return {name for name, is_static in last_is_static.items() if is_static} diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index b622c7c7..73c07f28 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -392,6 +392,20 @@ def assert_allowed(command: str) -> None: "M=P; M+=OST; M=GET; gh api repos/o/r/issues -X $M", "gh-api-method-value-appended-then-given-a-later-static-value-stays-allowed", ), + # False-positive guards for the thirty-first-round subshell-scoping + # fix (issue #1375): an ordinary, unrelated subshell elsewhere in the + # command must not spuriously deny a command whose watched name was + # never poisoned at all, and a harmless subshell assignment earlier + # in the command must not block a LATER, genuine top-level static + # reassignment from clearing poisoning normally. + ( + "TOOL=uv; VERB=safe; (echo hi); $TOOL $VERB foo", + "unrelated-harmless-subshell-alongside-a-never-poisoned-name-stays-allowed", + ), + ( + "TOOL=uv; (VERB=harmless); VERB=safe; $TOOL $VERB foo", + "real-top-level-static-clear-after-a-harmless-subshell-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1089,6 +1103,44 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "TOOL=uv; VERB=safe; VERB+=x; $TOOL $VERB foo", "var-split-tool-and-verb-given-a-static-value-then-appended-to-stays-denied", ), + # Found live by Step 8 independent review, thirty-first round (issue + # #1375): `_names_reassigned_from_a_static_value` and `_names_ + # cleared_by_a_later_static_reassignment` were both completely blind + # to bash's own subshell scoping -- a `(...)` grouping runs in a + # forked, isolated shell whose own assignments never propagate back + # to the parent, but both functions' own flat/per-segment scans + # treated a static assignment written INSIDE `(...)` exactly like an + # ordinary top-level one, letting it wrongly clear a genuinely + # poisoned name. Confirmed live via a stand-in `uv` binary on PATH + # that this genuinely runs `uv install foo`, NOT `safe foo` -- the + # parenthesized `VERB=safe` never reaches the parent shell's own + # `$VERB` at all. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); (VERB=safe); $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-subshell", + ), + # Same round, the append counterpart. + ( + "TOOL=uv; VERB=inst; VERB+=all; (VERB=safe); $TOOL $VERB foo", + "var-split-tool-and-verb-appended-then-cleared-via-a-subshell", + ), + # Same round, the array-element-assignment counterpart. + ( + "TOOL=uv; VERB=x; VERB[0]=install; (VERB=safe); $TOOL $VERB foo", + "var-split-tool-and-verb-array-element-assigned-then-cleared-via-a-subshell", + ), + # Same round, the `read` counterpart. + ( + "TOOL=uv; read VERB <<< install; (VERB=safe); $TOOL $VERB foo", + "var-split-tool-and-verb-read-into-then-cleared-via-a-subshell", + ), + # Same round, the gh-api-write counterpart: real bash genuinely runs + # `gh api repos/o/r/pulls/1/merge -X POST`, a genuine unreviewed + # write API call (e.g. merging a pull request). + ( + "M=safe; M=$(echo POST); (M=GET); gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-subshell", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 2f80ecd6..5d260b6b 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2606,6 +2606,60 @@ def test_names_reassigned_from_a_static_value_re_poisons_after_a_further_dynamic assert checker._names_reassigned_from_a_static_value(tokens) == {"M"} +def test_paren_depths_tracks_nesting() -> None: + """`_paren_depths` returns one depth per token, incrementing at each + `(` and decrementing at each `)`, never going negative.""" + tokens = ["A=1", "(", "B=2", "(", "C=3", ")", "D=4", ")", "E=5"] + assert checker._paren_depths(tokens) == [0, 1, 1, 2, 2, 2, 1, 1, 0] + + +def test_paren_depths_never_goes_negative_on_an_unmatched_close_paren() -> None: + assert checker._paren_depths([")", "A=1"]) == [0, 0] + + +def test_segment_tokens_with_subshell_depth_pairs_each_segment_with_its_own_depth() -> None: + tokens = ["TOOL=uv", ";", "(", "VERB=safe", ")", ";", "echo", "hi"] + result = checker._segment_tokens_with_subshell_depth(tokens) + assert result == [(["TOOL=uv"], 0), (["VERB=safe"], 1), (["echo", "hi"], 0)] + + +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_subshell_clear() -> None: + """CRITICAL bypass regression pin (round-31 independent review, issue + #1375): a static reassignment written INSIDE a `(...)` subshell + grouping never actually reaches the parent shell's own copy of the + name -- this function must not let it clear a genuinely poisoned + name.""" + tokens = ["VERB=harmless", "VERB=$(echo install)", "(", "VERB=safe", ")"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_allows_a_real_top_level_clear_after_a_harmless_subshell() -> None: + """No over-correction: an EARLIER, harmless subshell must not block a + LATER, genuine top-level static reassignment from clearing poisoning + normally.""" + tokens = ["VERB=harmless", "VERB=$(echo install)", "(", "VERB=whatever", ")", "VERB=safe"] + assert checker._names_reassigned_from_a_static_value(tokens) == set() + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, subshell_value=_VALUES) +def test_names_reassigned_from_a_static_value_matches_model_for_a_subshell_clear_attempt( + name: str, static_value: str, dynamic_value: str, subshell_value: str +) -> None: + """Model-based: for ANY identifier reassigned static -> dynamic and + then given a static value ONLY inside a `(...)` subshell, the name + stays poisoned regardless of the subshell's own assigned value -- + that assignment can never reach the parent scope.""" + tokens = [ + f"{name}={static_value}", + f"{name}=$({dynamic_value})", + "(", + f"{name}={subshell_value}", + ")", + ] + assert name in checker._names_reassigned_from_a_static_value(tokens) + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, final_static_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model_for_a_static_dynamic_static_sequence( @@ -2721,6 +2775,38 @@ def test_names_cleared_by_a_later_static_reassignment_ignores_an_unrelated_name( assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_subshell_assignment() -> None: + """CRITICAL bypass regression pin (round-31 independent review, issue + #1375): a static reassignment written INSIDE a `(...)` subshell + grouping never actually reaches the parent shell's own copy of the + name -- must not clear an append-poisoned candidate.""" + tokens = ["VERB=inst", "VERB+=all", "(", "VERB=safe", ")"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_still_clears_a_real_top_level_assignment_after_a_subshell() -> ( + None +): + """No over-correction: a harmless subshell earlier in the command + must not block a LATER, genuine top-level static reassignment from + clearing normally.""" + tokens = ["VERB=inst", "VERB+=all", "(", "VERB=whatever", ")", "VERB=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + +@_PROPERTIES +@given(name=_IDENTIFIERS, appended_value=_VALUES, subshell_value=_VALUES) +def test_names_cleared_by_a_later_static_reassignment_matches_model_for_a_subshell_clear_attempt( + name: str, appended_value: str, subshell_value: str +) -> None: + """Model-based: for ANY identifier appended to and then given a + static value ONLY inside a `(...)` subshell, the name is never + included in the cleared set -- that assignment can never reach the + parent scope.""" + tokens = [f"{name}=x", f"{name}+={appended_value}", "(", f"{name}={subshell_value}", ")"] + assert name not in checker._names_cleared_by_a_later_static_reassignment(tokens, {name}) + + def test_names_cleared_by_a_later_static_reassignment_clears_after_a_printf_v() -> None: tokens = ["printf", "-v", "VERB", "%s", "install", ";", "VERB=safe"] assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} @@ -2969,6 +3055,62 @@ def test_classify_still_denies_a_b1b_tool_and_verb_given_a_static_value_then_app assert verdict.deny is True +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_subshell_clear() -> None: + """CRITICAL bypass regression pin (round-31 independent review, + issue #1375): a static reassignment written INSIDE a `(...)` + subshell grouping never actually reaches the parent shell's own + copy of the name -- `_names_reassigned_from_a_static_value`'s own + pre-round-31 form let it wrongly clear a genuinely poisoned name. + Confirmed live via a stand-in `uv` binary on PATH that this + genuinely runs `uv install foo`, NOT `safe foo`.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); (VERB=safe); $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_subshell_clear() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`. + Confirmed live via a stand-in `gh` binary on PATH that this + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST`, a genuine + unreviewed write.""" + verdict = checker.classify("M=safe; M=$(echo POST); (M=GET); gh api repos/o/r/pulls/1/merge -X $M") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_appended_then_cleared_via_a_subshell() -> None: + """Same round, the append counterpart (`_names_cleared_by_a_later_ + static_reassignment`'s own subshell-blindness). Confirmed live via a + stand-in `uv` binary on PATH that this genuinely runs `uv install + foo`, NOT `safe foo`.""" + verdict = checker.classify("TOOL=uv; VERB=inst; VERB+=all; (VERB=safe); $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_array_element_assigned_then_cleared_via_a_subshell() -> None: + verdict = checker.classify("TOOL=uv; VERB=x; VERB[0]=install; (VERB=safe); $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_read_into_then_cleared_via_a_subshell() -> None: + verdict = checker.classify("TOOL=uv; read VERB <<< install; (VERB=safe); $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_allows_an_unrelated_harmless_subshell_alongside_a_never_poisoned_name() -> None: + """No over-correction: an ordinary, unrelated subshell elsewhere in + the command must not spuriously deny a command whose watched name + was never poisoned at all.""" + verdict = checker.classify("TOOL=uv; VERB=safe; (echo hi); $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_subshell() -> None: + """No over-correction: a harmless subshell assignment earlier in the + command must not block a LATER, genuine top-level static + reassignment from clearing poisoning normally.""" + verdict = checker.classify("TOOL=uv; (VERB=harmless); VERB=safe; $TOOL $VERB foo") + assert verdict.deny is False + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 82f279e7888f73dac02b916dd11aec736a929e89 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 01:57:09 +0000 Subject: [PATCH 37/46] fix(hooks): generalize scope-isolation to pipe stages, background jobs, and local declarations A fresh, independent adversarial review of this PR's current head (round 32) found that round 31's own subshell-scoping fix covered only `(...)` grouping, leaving three sibling bash constructs that equally isolate an assignment from the parent shell's own scope exploitable via the identical trick: wrap a fake "the value is safe now" reassignment in one of them to escape a genuine poisoning a prior append/read/ array-element/dynamic reassignment already established. Independently reproduced live before acting, via `classify()` and real bash execution with stand-in `uv`/`gh` binaries on PATH, for all three constructs: `TOOL=uv; VERB=harmless; VERB=$(echo install); true | VERB=safe; $TOOL $VERB foo` resolved to `deny=False` even though real bash genuinely runs `uv install foo`, NOT `safe foo` (every pipeline stage forks its own subshell by default, including the last one, absent `shopt -s lastpipe`); `VERB=safe & wait` reproduces identically for a backgrounded job (`cmd &` forks a subshell for `cmd` alone); `f() { local VERB=safe; }; f` reproduces identically for a function-local declaration (bash functions do NOT get their own variable scope by default -- a plain `VERB=safe` inside a function body genuinely WOULD leak to the caller -- but `local` explicitly opts a single assignment out of that leak, and the pre-round-32 code had no concept of `local` at all). All three reproduce identically for `_rule_gh_api_write` too (e.g. real bash genuinely running an unreviewed `-X POST` pull-request merge). Confirmed through the real wrapper: all six (three constructs x two consumers) now deny with exit 2 where they previously allowed with exit 0. Closed by generalizing round 31's own `_paren_depths`/`_segment_tokens_ with_subshell_depth` (subshell-only) into `_raw_segments_with_ boundaries` and `_segment_tokens_with_scope_isolation`, which mark a segment isolated when its own paren-nesting depth is 1+, OR it is any stage of a `|` pipeline (via its own terminating/preceding boundary token), OR it is itself backgrounded via a trailing `&` (only the segment BEFORE `&` is isolated -- what follows runs in the parent shell as normal), OR it contains the literal `local` keyword anywhere (not necessarily first, since a function body's own `{` shares the same segment). `_names_reassigned_from_a_static_value` and `_names_cleared_ by_a_later_static_reassignment` now consult this generalized isolation flag instead of the round-31 paren-depth-only one. A fourth candidate -- bash's `((EXPR))` arithmetic-command syntax, which tokenizes identically to a deliberately-spaced, genuinely double-nested `( (cmd) )` subshell grouping in this classifier's own tokenizer (confirmed directly: both produce the byte-identical token sequence, since the tokenizer already discards the space real bash's own lexer uses to disambiguate them) -- was investigated and its "obvious" fix deliberately REJECTED after live verification proved it unsafe: real bash's arithmetic evaluation can only ever assign a NUMBER to the target name (`((VERB=safe))` sets `$VERB` to `"0"`, not the string "safe"), but the double-subshell reading of the identical tokens can assign an arbitrary STRING that stays fully isolated from the parent (`( (VERB=totallysafe) )` leaves the parent's own `$VERB` completely unchanged, confirmed live). Treating the ambiguous pair as depth-neutral would have let this classifier's own literal-text extraction trust that string and wrongly clear a genuine poisoning -- a NEW bypass in exchange for fixing an over-denial. The over-denial on genuine `((...))` arithmetic usage is therefore kept, deliberately NOT fixed, disclosed in `_segment_tokens_with_scope_isolation`'s own docstring and pinned by regression tests at every layer so it cannot regress silently into the unsafe direction. Regression tests added at every established layer: unit tests for `_raw_segments_with_boundaries`/`_segment_tokens_with_scope_isolation` directly (pipe-stage, background-job, and local-declaration isolation, plus the no-over-correction cases), `@given` Hypothesis property tests (including one exercising `_segment_tokens_with_scope_isolation` itself by name, per issue #1178's own coverage requirement), unit and `@given` tests for both fixed functions' new isolation classes, `classify()`-level end-to-end pins for every reported bypass shape plus false-positive controls and the disclosed arithmetic/double-subshell residual, and wrapper-level end-to-end pins against real stand-in `uv`/`gh` binaries (hooks/test_gitapex_check_bash_safety.py, tests/ test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI's own whole-codebase invocation), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-32 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 274 +++++++++++------ hooks/test_gitapex_check_bash_safety.py | 71 +++++ ...st_gitapex_check_bash_safety_properties.py | 279 +++++++++++++++++- 3 files changed, 528 insertions(+), 96 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index f35c7dd7..944338d4 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4073,61 +4073,156 @@ def _names_reassigned_by_untracked_construct(tokens: list[str]) -> set[str]: return names -def _paren_depths(tokens: list[str]) -> list[int]: - """The `(...)`-subshell nesting depth of each token in TOKENS, as a - list the same length and order as TOKENS itself (0 = top-level - scope, 1+ = inside one or more subshell groupings). `(`/`)` - themselves each get the depth in effect AT that boundary token (the - new, deeper depth for `(`; the old, shallower depth for `)`) -- - irrelevant in practice since a caller only ever consults the depth - of an actual assignment-shaped content token, never a bare `(`/`)`. - - Added by round 31 (issue #1375): `segment_tokens` treats `(`/`)` - as pure sequencing boundaries with no depth tracking at all -- - correct for simple-command splitting, but insufficient for a caller - that needs to know whether a given assignment can actually affect a - reference OUTSIDE its own subshell scope. Real bash's own subshell - semantics: a `(...)` grouping runs in a forked, isolated shell whose - own assignments never propagate back to the parent -- see - `_names_reassigned_from_a_static_value`'s own docstring for the live - bypass this closes.""" - depths: list[int] = [] - depth = 0 - for tok in tokens: - if tok == "(": - depth += 1 - depths.append(depth) - elif tok == ")": - depths.append(depth) - depth = max(depth - 1, 0) - else: - depths.append(depth) - return depths - - -def _segment_tokens_with_subshell_depth(tokens: list[str]) -> list[tuple[list[str], int]]: - """Like `segment_tokens`, but pairs each returned segment with its - own `(...)`-subshell nesting DEPTH (see `_paren_depths`'s own - docstring) at the point it appears in TOKENS. A single segment's own - depth is always internally consistent -- `(`/`)` are pure segment - boundaries in `segment_tokens` too, so no segment can itself - straddle a depth change -- added by round 31 (issue #1375) for - `_names_cleared_by_a_later_static_reassignment`'s own use; see that - function's own docstring for the live bypass this closes.""" - segments: list[tuple[list[str], int]] = [([], 0)] +def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], int, str | None]]: + """Every segment TOKENS splits into (including EMPTY ones, unlike + `segment_tokens`'s own filtered output -- an empty segment, e.g. + between `(` and `)` in a parameterless `f()` function definition, + still needs to occupy its own position in this list so a caller can + look at its neighbors), each paired with its own `(...)`-nesting + depth (see the round-31 paragraph in `_names_reassigned_from_a_ + static_value`'s own docstring) and the single boundary token that + TERMINATES it (`None` for the final segment, which runs to the end + of TOKENS with no terminator). Building block for `_segment_tokens_ + with_scope_isolation`'s own pipe/background-job detection, which + needs to know each segment's own NEIGHBORING boundary tokens, not + just its depth.""" + segments: list[list[str]] = [[]] + seg_depths: list[int] = [0] + terminators: list[str | None] = [] depth = 0 for tok in tokens: if tok == "(": + terminators.append(tok) depth += 1 - segments.append(([], depth)) + segments.append([]) + seg_depths.append(depth) elif tok == ")": + terminators.append(tok) depth = max(depth - 1, 0) - segments.append(([], depth)) + segments.append([]) + seg_depths.append(depth) elif tok in _SINGLE_OPS or tok in _MULTI_OPS: - segments.append(([], depth)) + terminators.append(tok) + segments.append([]) + seg_depths.append(depth) else: - segments[-1][0].append(tok) - return [(seg, d) for seg, d in segments if seg] + segments[-1].append(tok) + terminators.append(None) + return list(zip(segments, seg_depths, terminators, strict=True)) + + +def _segment_tokens_with_scope_isolation(tokens: list[str]) -> list[tuple[list[str], bool]]: + """Like `segment_tokens`, but pairs each returned (non-empty) + segment with whether it is ISOLATED from the parent shell's own + scope -- unable to affect a reference outside itself, so any plain + static assignment inside it must never be trusted to CLEAR an + earlier poisoning (though a dynamic/append/array-element/read/ + printf-v event inside it may still POISON, staying conservative is + always the safe direction). A segment is isolated when: + + - its own `(...)`-subshell nesting depth is 1+ (round 31's own + mechanism, generalized here rather than replaced); + - it is any stage of a `|` pipeline -- bash forks a SEPARATE + subshell for EVERY pipeline stage by default, INCLUDING THE LAST + ONE, absent `shopt -s lastpipe` (which this classifier has no way + to know is set), so a caller must never trust a pipe stage's own + assignment regardless of its position in the pipe chain; + - it is itself backgrounded via a trailing `&` (the segment BEFORE + `&` forks into its own subshell; the segment AFTER `&` runs in the + parent shell as normal and is NOT isolated by that `&` alone -- + only what precedes it is); + - it contains the literal `local` keyword as one of its own tokens + anywhere, not necessarily first -- a function body's own opening + `{` (not itself a `segment_tokens` boundary, so it shares the + SAME segment as the `local` statement that follows it, e.g. + `["{", "local", "VERB=safe"]` for `f() { local VERB=safe; }`) + would otherwise sit at position 0 and hide the keyword from a + first-token-only check (bash's own function-local variable + scoping: a `local NAME=value` inside a function body never leaks + to the function's caller, unlike a plain `NAME=value` in the same + position, which WOULD leak). + + CRITICAL bypasses found by independent adversarial review (round + 32, issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: round 31's own `_paren_depths`/`_segment_tokens_ + with_subshell_depth` covered ONLY `(...)` subshell grouping, leaving + three SIBLING scope-isolating constructs equally exploitable via the + identical trick -- wrap a fake "the value is safe now" reassignment + in one of them to escape a genuine poisoning a prior append/read/ + array-element/dynamic reassignment already established. + `TOOL=uv; VERB=harmless; VERB=$(echo install); true | VERB=safe; + $TOOL $VERB foo` resolved to `deny=False` even though real bash + genuinely runs `uv install foo`, NOT `safe foo` (confirmed live via + a stand-in `uv` binary on PATH: captured argv `uv called with: + install foo`) -- the pipe stage's own `VERB=safe` never reaches the + parent shell. `TOOL=uv; VERB=harmless; VERB=$(echo install); + VERB=safe & wait; $TOOL $VERB foo` reproduces identically for a + backgrounded job (captured argv: `uv called with: install foo`, + NOT `safe foo`). `TOOL=uv; VERB=harmless; VERB=$(echo install); f() + { local VERB=safe; }; f; $TOOL $VERB foo` reproduces identically for + a function-local reassignment (captured argv: `uv called with: + install foo`) -- bash functions do NOT get their own variable scope + by default (a plain `VERB=safe` inside a function body genuinely + WOULD leak to the caller), but `local` explicitly opts a single + assignment out of that leak, and the pre-round-32 code had no + concept of `local` at all, so it trusted the assignment exactly like + an ordinary top-level one. All three reproduce identically for + `_rule_gh_api_write` too (e.g. `M=safe; M=$(echo POST); true | M=GET; + gh api repos/o/r/pulls/1/merge -X $M` -- real bash genuinely runs + `-X POST`, a genuine unreviewed write). Confirmed through the real + wrapper: all six (three constructs x two consumers) now deny with + exit 2 where they previously allowed with exit 0. + + A FOURTH candidate was investigated and deliberately NOT folded into + this isolation model: bash's `((EXPR))` arithmetic-command syntax + also uses two characters that individually tokenize as bare `(`/`)` + in this file's own tokenizer (confirmed directly: `tokenize()` + produces the IDENTICAL token sequence `['(', '(', 'VERB=1', ')', + ')']` for BOTH `((VERB=1))` and the deliberately-spaced `( (VERB=1) + )` -- real bash's own lexer distinguishes the two by the ABSENCE or + PRESENCE of a space between the parens, a distinction this + classifier's tokenizer already discards before this function ever + runs), so `((VERB=1))` is currently treated as TWO nested subshells + (depth 2), denying a harmless, genuinely-numeric arithmetic + assignment (`TOOL=uv; VERB=harmless; VERB=$(echo install); + ((VERB=1)); $TOOL $VERB foo` -- real bash genuinely sets `$VERB` to + the number `1`, never a watched verb) -- but the naive fix (treating + adjacent `((`/`))` as depth-neutral, since REAL arithmetic never + leaks a non-numeric value to the parent) was independently tried and + REJECTED after live verification proved it unsafe: `((VERB=safe))` + (a non-numeric RHS) does NOT set `$VERB` to + the string "safe" in real bash at all -- arithmetic context + evaluates the bare word "safe" as an (unset) variable reference, + defaulting to `0`, so `$VERB` becomes `"0"`, never a watched-verb- + matching string -- but the deliberately-spaced double-subshell + `( (VERB=totallysafe) )`, which this tokenizer cannot distinguish + from `((VERB=totallysafe))` at all, genuinely assigns the STRING + "totallysafe" INSIDE the isolated inner subshell while leaving the + parent's own `$VERB` completely untouched (confirmed live: `TOOL=uv; + VERB=harmless; VERB=$(echo install); ( (VERB=totallysafe) ); + echo "VERB is now: $VERB"` prints `VERB is now: install` -- the + parent's own poisoned value survives unchanged). Treating the + ambiguous adjacent-paren pair as depth-neutral would have let THIS + classifier's own literal-text extraction read "totallysafe" as + VERB's new, trusted value and clear the poisoning -- a NEW security + bypass, reopening exactly the class of defect this whole function + exists to close, in exchange for fixing an over-denial. The + over-denial on genuine `((...))` arithmetic usage is therefore kept, + deliberately NOT fixed -- the same deny-on-genuine-ambiguity posture + this file has followed in every prior round; closing it soundly + would require the upstream tokenizer to preserve the presence/ + absence of a space between adjacent parens, which is out of this + round's scope.""" + raw = _raw_segments_with_boundaries(tokens) + result: list[tuple[list[str], bool]] = [] + for i, (seg, depth, terminator) in enumerate(raw): + if not seg: + continue + preceding = raw[i - 1][2] if i > 0 else None + isolated = depth > 0 or terminator in ("|", "&") or preceding == "|" or "local" in seg + result.append((seg, isolated)) + return result def _names_reassigned_from_a_static_value(tokens: list[str]) -> set[str]: @@ -4243,31 +4338,34 @@ def _names_reassigned_from_a_static_value(tokens: list[str]) -> set[str]: (M=GET); gh api repos/o/r/pulls/1/merge -X $M` also resolved to `deny=False` -- real bash genuinely runs `-X POST` (captured argv confirms it), a genuine unreviewed write. Closed by tracking each - assignment token's own `(...)`-nesting depth via `_paren_depths` and - only letting a static reassignment clear POISONED when it occurs at - depth 0 (true top-level scope, the only scope whose own reassignment - can actually affect a top-level reference); a static assignment - found at depth 1+ is treated as invisible to the parent scope -- - neither clearing nor registering EVER_STATIC -- while a DYNAMIC - reassignment at ANY depth still poisons unconditionally, since - staying conservative about what a subshell might have done is always - the safe direction, never the dangerous one. Checkout/restore's own - `_names_with_dynamic_assignment` was independently confirmed - unaffected by this exact bypass shape (it never clears at all, - regardless of depth, so it already denied this shape correctly).""" + assignment token's own scope isolation (originally just `(...)`- + nesting depth via `_paren_depths`, GENERALIZED by round 32 -- see + `_segment_tokens_with_scope_isolation`'s own docstring -- to also + cover pipe stages, backgrounded jobs, and `local` scoping) and only + letting a static reassignment clear POISONED when it occurs in a + non-isolated (true top-level) segment; an isolated one is treated as + invisible to the parent scope -- neither clearing nor registering + EVER_STATIC -- while a DYNAMIC reassignment in ANY segment still + poisons unconditionally, since staying conservative about what an + isolated context might have done is always the safe direction, never + the dangerous one. Checkout/restore's own `_names_with_dynamic_ + assignment` was independently confirmed unaffected by this exact + bypass shape (it never clears at all, regardless of scope, so it + already denied this shape correctly).""" ever_static: set[str] = set() poisoned: set[str] = set() - for tok, depth in zip(tokens, _paren_depths(tokens), strict=True): - match = _ASSIGN_RE.match(tok) - if not match: - continue - name = match.group(1) - if _is_dynamic(tok): - if name in ever_static: - poisoned.add(name) - elif depth == 0: - ever_static.add(name) - poisoned.discard(name) + for seg, isolated in _segment_tokens_with_scope_isolation(tokens): + for tok in seg: + match = _ASSIGN_RE.match(tok) + if not match: + continue + name = match.group(1) + if _is_dynamic(tok): + if name in ever_static: + poisoned.add(name) + elif not isolated: + ever_static.add(name) + poisoned.discard(name) return poisoned @@ -4338,21 +4436,24 @@ def _names_cleared_by_a_later_static_reassignment(tokens: list[str], candidates: `uv` binary on PATH: captured argv `uv called with: install foo`); the `read`- and array-element-reassigned counterparts reproduce identically. Closed the same way as `_names_reassigned_from_a_ - static_value`'s own round-31 fix: each segment's own `(...)`-nesting - depth (via `_segment_tokens_with_subshell_depth`) gates whether a - plain static assignment in that segment may set a candidate's own - LAST-event state to "cleared" -- only a depth-0 (true top-level) - static assignment may do so; a depth-1+ one is treated as invisible - to the parent scope, leaving whatever state a prior event already - recorded untouched. Every OTHER event class (append, array-element, - read/printf-v, and a dynamic plain assignment) still marks the - candidate "not cleared" regardless of depth, since staying - conservative about what a subshell might have done is always the - safe direction, never the dangerous one.""" + static_value`'s own round-31 fix: each segment's own scope isolation + (originally just `(...)`-nesting depth via `_segment_tokens_with_ + subshell_depth`, GENERALIZED by round 32 -- see `_segment_tokens_ + with_scope_isolation`'s own docstring -- to also cover pipe stages, + backgrounded jobs, and `local` scoping) gates whether a plain static + assignment in that segment may set a candidate's own LAST-event + state to "cleared" -- only a non-isolated (true top-level) static + assignment may do so; an isolated one is treated as invisible to the + parent scope, leaving whatever state a prior event already recorded + untouched. Every OTHER event class (append, array-element, read/ + printf-v, and a dynamic plain assignment) still marks the candidate + "not cleared" regardless of scope, since staying conservative about + what an isolated context might have done is always the safe + direction, never the dangerous one.""" if not candidates: return set() last_is_static: dict[str, bool] = {} - for seg, depth in _segment_tokens_with_subshell_depth(tokens): + for seg, isolated in _segment_tokens_with_scope_isolation(tokens): if seg and not _is_dynamic(seg[0]): head = seg[0].lower() if head in _READ_COMMAND_WORDS: @@ -4385,11 +4486,12 @@ def _names_cleared_by_a_later_static_reassignment(tokens: list[str], candidates: continue if _is_dynamic(token): last_is_static[name] = False - elif depth == 0: + elif not isolated: last_is_static[name] = True - # depth > 0 and static: a subshell-scoped reassignment - # never reaches the parent scope -- leave any existing - # state untouched (round 31). + # isolated and static: a scope-isolated reassignment + # (subshell/pipe-stage/background job/`local`) never + # reaches the parent scope -- leave any existing state + # untouched (round 31/32). return {name for name, is_static in last_is_static.items() if is_static} diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 73c07f28..f0555c97 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -406,6 +406,33 @@ def assert_allowed(command: str) -> None: "TOOL=uv; (VERB=harmless); VERB=safe; $TOOL $VERB foo", "real-top-level-static-clear-after-a-harmless-subshell-stays-allowed", ), + # False-positive guards for the thirty-second-round scope-isolation + # generalization (issue #1375): an ordinary, unrelated pipeline, + # background job, or function call elsewhere in the command must not + # spuriously deny a command whose watched name was never poisoned at + # all, and a harmless pipeline earlier in the command must not block + # a LATER, genuine top-level static reassignment from clearing + # poisoning normally. + ( + "TOOL=uv; VERB=safe; true | cat; $TOOL $VERB foo", + "unrelated-harmless-pipeline-alongside-a-never-poisoned-name-stays-allowed", + ), + ( + "TOOL=uv; VERB=safe; sleep 0 & wait; $TOOL $VERB foo", + "unrelated-harmless-background-job-alongside-a-never-poisoned-name-stays-allowed", + ), + ( + "TOOL=uv; f() { echo hi; }; f; VERB=safe; $TOOL $VERB foo", + "unrelated-harmless-function-call-alongside-a-never-poisoned-name-stays-allowed", + ), + ( + "TOOL=uv; g() { VERB=notlocal; }; VERB=safe; $TOOL $VERB foo", + "unrelated-function-bodys-own-plain-non-local-assignment-does-not-interfere", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); true | echo hi; VERB=safe; $TOOL $VERB foo", + "real-top-level-static-clear-after-a-harmless-pipeline-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1141,6 +1168,50 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "M=safe; M=$(echo POST); (M=GET); gh api repos/o/r/pulls/1/merge -X $M", "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-subshell", ), + # Found live by Step 8 independent review, thirty-second round (issue + # #1375): round 31's own subshell-scoping fix covered ONLY `(...)` + # grouping, leaving three sibling scope-isolating bash constructs + # equally exploitable via the identical trick. Confirmed live via a + # stand-in `uv`/`gh` binary on PATH that each genuinely runs the + # dangerous command, NOT the parenthesized/piped/backgrounded/local + # distractor value. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); true | VERB=safe; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-pipe-stage", + ), + ( + "M=safe; M=$(echo POST); true | M=GET; gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-pipe-stage", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); VERB=safe & wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-background-job", + ), + ( + "M=safe; M=$(echo POST); M=GET & wait; gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-background-job", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); f() { local VERB=safe; }; f; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-local-declaration", + ), + ( + "M=safe; M=$(echo POST); f() { local M=GET; }; f; gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-local-declaration", + ), + # Same round, the disclosed arithmetic-vs-double-subshell ambiguity + # residual: `((VERB=1))` stays denied on purpose (see `_segment_ + # tokens_with_scope_isolation`'s own docstring for why the naive fix + # was rejected as unsafe), and a genuinely double-nested, spaced + # subshell carrying a distractor value must stay denied too. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); ((VERB=1)); $TOOL $VERB foo", + "arithmetic-double-paren-content-stays-denied-as-a-disclosed-residual", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); ( (VERB=totallysafe) ); $TOOL $VERB foo", + "deliberately-spaced-double-subshell-distractor-stays-denied", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 5d260b6b..5d83d997 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2606,21 +2606,78 @@ def test_names_reassigned_from_a_static_value_re_poisons_after_a_further_dynamic assert checker._names_reassigned_from_a_static_value(tokens) == {"M"} -def test_paren_depths_tracks_nesting() -> None: - """`_paren_depths` returns one depth per token, incrementing at each - `(` and decrementing at each `)`, never going negative.""" - tokens = ["A=1", "(", "B=2", "(", "C=3", ")", "D=4", ")", "E=5"] - assert checker._paren_depths(tokens) == [0, 1, 1, 2, 2, 2, 1, 1, 0] +def test_raw_segments_with_boundaries_tracks_nesting_and_terminators() -> None: + """`_raw_segments_with_boundaries` returns one (segment, depth, + terminator) triple per segment, including empty ones, with depth + incrementing at each `(` and decrementing at each `)` (never + negative), and each non-final entry's own terminator naming the + boundary token that ended it.""" + tokens = ["A=1", "(", "B=2", ")", "C=3"] + result = checker._raw_segments_with_boundaries(tokens) + assert result == [ + (["A=1"], 0, "("), + (["B=2"], 1, ")"), + (["C=3"], 0, None), + ] -def test_paren_depths_never_goes_negative_on_an_unmatched_close_paren() -> None: - assert checker._paren_depths([")", "A=1"]) == [0, 0] +def test_raw_segments_with_boundaries_never_goes_negative_on_an_unmatched_close_paren() -> None: + result = checker._raw_segments_with_boundaries([")", "A=1"]) + assert result == [([], 0, ")"), (["A=1"], 0, None)] -def test_segment_tokens_with_subshell_depth_pairs_each_segment_with_its_own_depth() -> None: +def test_segment_tokens_with_scope_isolation_marks_subshell_content_isolated() -> None: tokens = ["TOOL=uv", ";", "(", "VERB=safe", ")", ";", "echo", "hi"] - result = checker._segment_tokens_with_subshell_depth(tokens) - assert result == [(["TOOL=uv"], 0), (["VERB=safe"], 1), (["echo", "hi"], 0)] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["TOOL=uv"], False), (["VERB=safe"], True), (["echo", "hi"], False)] + + +def test_segment_tokens_with_scope_isolation_marks_every_pipe_stage_isolated() -> None: + """Every stage of a `|` pipeline is isolated, including the LAST + one -- bash forks a subshell for each stage by default, absent + `shopt -s lastpipe`, which this classifier has no way to know is + set.""" + tokens = ["cmd1", "|", "cmd2", "|", "cmd3"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["cmd1"], True), (["cmd2"], True), (["cmd3"], True)] + + +def test_segment_tokens_with_scope_isolation_marks_only_the_backgrounded_segment_isolated() -> None: + """`cmd1 & cmd2` backgrounds ONLY cmd1 -- cmd2 runs in the parent + shell as normal immediately afterward, so it must NOT be marked + isolated merely for following a `&`.""" + tokens = ["cmd1", "&", "cmd2"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["cmd1"], True), (["cmd2"], False)] + + +def test_segment_tokens_with_scope_isolation_marks_a_local_declaration_isolated() -> None: + """A segment containing the literal `local` keyword anywhere (not + necessarily first -- a function body's own opening `{` shares the + same segment) is isolated.""" + tokens = ["{", "local", "VERB=safe"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["{", "local", "VERB=safe"], True)] + + +def test_segment_tokens_with_scope_isolation_leaves_ordinary_segments_unisolated() -> None: + tokens = ["VERB=inst", ";", "VERB+=all", ";", "VERB=safe"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["VERB=inst"], False), (["VERB+=all"], False), (["VERB=safe"], False)] + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_VALUES) +def test_segment_tokens_with_scope_isolation_matches_model_for_a_pipe_stage(name: str, value: str) -> None: + """Model-based, exercising `_segment_tokens_with_scope_isolation` + directly (issue #1178's own detection-logic property-coverage + requirement): for ANY identifier assigned a value as the SECOND + stage of a two-stage `|` pipeline, that segment is marked isolated + -- and the FIRST, ordinary top-level segment preceding the whole + pipeline is not.""" + tokens = [f"{name}=first", ";", "true", "|", f"{name}={value}"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [([f"{name}=first"], False), (["true"], True), ([f"{name}={value}"], True)] def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_subshell_clear() -> None: @@ -2660,6 +2717,50 @@ def test_names_reassigned_from_a_static_value_matches_model_for_a_subshell_clear assert name in checker._names_reassigned_from_a_static_value(tokens) +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_pipe_stage_clear() -> None: + """CRITICAL bypass regression pin (round-32 independent review, + issue #1375): every stage of a `|` pipeline runs in its OWN forked + subshell by default (including the LAST stage, absent `shopt -s + lastpipe`) -- a static reassignment inside any stage can never reach + the parent shell's own copy of the name.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "true", "|", "VERB=safe"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_background_job_clear() -> None: + """Same round, the backgrounded-job counterpart: `cmd &` forks a + subshell for `cmd` alone.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "VERB=safe", "&", "wait"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_local_declaration_clear() -> None: + """Same round, the function-local counterpart: bash functions do NOT + get their own variable scope by default, but `local` explicitly + opts a single assignment out of leaking to the caller.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "{", "local", "VERB=safe", "}"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_allows_a_real_top_level_clear_after_a_harmless_pipeline() -> None: + """No over-correction: an EARLIER, harmless pipeline must not block a + LATER, genuine top-level static reassignment from clearing normally.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "true", "|", "echo", "hi", ";", "VERB=safe"] + assert checker._names_reassigned_from_a_static_value(tokens) == set() + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, isolated_value=_VALUES) +def test_names_reassigned_from_a_static_value_matches_model_for_a_pipe_stage_clear_attempt( + name: str, static_value: str, dynamic_value: str, isolated_value: str +) -> None: + """Model-based: for ANY identifier reassigned static -> dynamic and + then given a static value ONLY inside a pipe stage, the name stays + poisoned regardless of the pipe stage's own assigned value.""" + tokens = [f"{name}={static_value}", ";", f"{name}=$({dynamic_value})", ";", "true", "|", f"{name}={isolated_value}"] + assert name in checker._names_reassigned_from_a_static_value(tokens) + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, final_static_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model_for_a_static_dynamic_static_sequence( @@ -2807,6 +2908,47 @@ def test_names_cleared_by_a_later_static_reassignment_matches_model_for_a_subshe assert name not in checker._names_cleared_by_a_later_static_reassignment(tokens, {name}) +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_pipe_stage() -> None: + """CRITICAL bypass regression pin (round-32 independent review, + issue #1375): a static reassignment inside any pipe stage never + reaches the parent shell's own copy of the name -- must not clear + an append-poisoned candidate.""" + tokens = ["VERB=inst", "VERB+=all", ";", "true", "|", "VERB=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_background_job() -> None: + tokens = ["VERB=inst", "VERB+=all", ";", "VERB=safe", "&", "wait"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_local_declaration() -> None: + tokens = ["VERB=inst", "VERB+=all", ";", "{", "local", "VERB=safe", "}"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_still_clears_a_real_top_level_assignment_after_a_pipeline() -> ( + None +): + """No over-correction: a harmless pipeline earlier in the command + must not block a LATER, genuine top-level static reassignment from + clearing normally.""" + tokens = ["VERB=inst", "VERB+=all", ";", "true", "|", "echo", "hi", ";", "VERB=safe"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + +@_PROPERTIES +@given(name=_IDENTIFIERS, appended_value=_VALUES, isolated_value=_VALUES) +def test_names_cleared_by_a_later_static_reassignment_matches_model_for_a_pipe_stage_clear_attempt( + name: str, appended_value: str, isolated_value: str +) -> None: + """Model-based: for ANY identifier appended to and then given a + static value ONLY inside a pipe stage, the name is never included + in the cleared set.""" + tokens = [f"{name}=x", f"{name}+={appended_value}", ";", "true", "|", f"{name}={isolated_value}"] + assert name not in checker._names_cleared_by_a_later_static_reassignment(tokens, {name}) + + def test_names_cleared_by_a_later_static_reassignment_clears_after_a_printf_v() -> None: tokens = ["printf", "-v", "VERB", "%s", "install", ";", "VERB=safe"] assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} @@ -3111,6 +3253,123 @@ def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_subshell assert verdict.deny is False +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_pipe_stage_clear() -> None: + """CRITICAL bypass regression pin (round-32 independent review, + issue #1375): every stage of a `|` pipeline runs in its OWN forked + subshell by default (including the LAST stage, absent `shopt -s + lastpipe`), so a static reassignment inside a pipe stage never + reaches the parent shell's own copy of the name. Confirmed live via + a stand-in `uv` binary on PATH that this genuinely runs `uv install + foo`, NOT `safe foo`.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); true | VERB=safe; $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_pipe_stage_clear() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`. + Confirmed live via a stand-in `gh` binary on PATH that this + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST`, a genuine + unreviewed write.""" + verdict = checker.classify("M=safe; M=$(echo POST); true | M=GET; gh api repos/o/r/pulls/1/merge -X $M") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_background_job_clear() -> None: + """Same round, the backgrounded-job counterpart: `cmd &` forks a + subshell for `cmd` alone. Confirmed live via a stand-in `uv` binary + on PATH that this genuinely runs `uv install foo`, NOT `safe foo`.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); VERB=safe & wait; $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_background_job_clear() -> None: + verdict = checker.classify("M=safe; M=$(echo POST); M=GET & wait; gh api repos/o/r/pulls/1/merge -X $M") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_local_declaration_clear() -> None: + """Same round, the function-local counterpart: bash functions do NOT + get their own variable scope by default, but `local` explicitly + opts a single assignment out of leaking to the caller. Confirmed + live via a stand-in `uv` binary on PATH that this genuinely runs `uv + install foo`, NOT `safe foo`.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); f() { local VERB=safe; }; f; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_local_declaration_clear() -> None: + verdict = checker.classify("M=safe; M=$(echo POST); f() { local M=GET; }; f; gh api repos/o/r/pulls/1/merge -X $M") + assert verdict.deny is True + + +def test_classify_allows_an_unrelated_harmless_pipeline_alongside_a_never_poisoned_name() -> None: + """No over-correction: an ordinary, unrelated pipeline elsewhere in + the command must not spuriously deny a command whose watched name + was never poisoned at all.""" + verdict = checker.classify("TOOL=uv; VERB=safe; true | cat; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_an_unrelated_harmless_background_job_alongside_a_never_poisoned_name() -> None: + verdict = checker.classify("TOOL=uv; VERB=safe; sleep 0 & wait; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_an_unrelated_harmless_function_call_alongside_a_never_poisoned_name() -> None: + verdict = checker.classify("TOOL=uv; f() { echo hi; }; f; VERB=safe; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_pipeline() -> None: + """No over-correction: a harmless pipeline earlier in the command + must not block a LATER, genuine top-level static reassignment from + clearing poisoning normally.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); true | echo hi; VERB=safe; $TOOL $VERB foo" + ) + assert verdict.deny is False + + +def test_classify_denies_arithmetic_double_paren_content_as_a_disclosed_residual() -> None: + """DISCLOSED, deliberately NOT fixed (round 32, issue #1375): this + classifier's own tokenizer produces the IDENTICAL token sequence for + bash's `((EXPR))` arithmetic-command syntax and a deliberately- + spaced, genuinely double-nested `( (cmd) )` subshell grouping, + discarding the space that real bash's own lexer uses to disambiguate + them. Treating adjacent `((`/`))` as depth-neutral (the "obviously + correct" fix for the resulting over-denial on harmless arithmetic + usage) was independently tried and REJECTED: real bash's own + arithmetic evaluation can only ever assign a NUMBER to the target + name (confirmed live: `((VERB=safe))` sets `$VERB` to `"0"`, not the + string "safe" -- bash reads the bare word as an unset variable + reference), but the genuinely-double-nested-subshell reading of the + SAME tokens can assign an ARBITRARY STRING that stays fully isolated + from the parent scope (confirmed live: `( (VERB=totallysafe) )` + leaves the parent's own `$VERB` completely unchanged). Treating the + ambiguous pair as depth-neutral would let this classifier's own + literal-text extraction trust that arbitrary string as VERB's new + value and wrongly clear a genuine poisoning -- a NEW bypass, reopening + exactly the class of defect `_names_reassigned_from_a_static_value` + exists to close. The over-denial on genuine arithmetic usage is kept + deliberately, matching this file's own established deny-on- + ambiguity posture; closing it soundly would require the upstream + tokenizer to preserve the presence/absence of a space between + adjacent parens.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); ((VERB=1)); $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_deliberately_spaced_double_subshell_distractor() -> None: + """No under-correction: confirms the rejected "arithmetic-neutral" + fix was correctly NOT applied -- a genuinely double-nested, spaced + subshell carrying a plausible-looking distractor value must stay + denied, since real bash's own parent scope is unaffected by it.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); ( (VERB=totallysafe) ); $TOOL $VERB foo") + assert verdict.deny is True + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 5002c79cc2ebccc900d7ca9534e44943844dfe03 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 02:13:16 +0000 Subject: [PATCH 38/46] fix(hooks): recognize process substitution and declare/typeset as scope-isolating A fresh, independent adversarial review of this PR's current head (round 33) found two more sibling gaps in the same scope-isolation area rounds 31 and 32 already closed for `(...)` subshells, pipe stages, background jobs, and `local` declarations. Process substitution (`<(cmd)`/`>(cmd)`) runs its own content in a separate, isolated subshell, exactly like `$(...)` command substitution -- but this classifier's own tokenizer fuses `<(`/`>(` into their own distinct tokens, never a bare `(`, so `_raw_segments_with_boundaries`'s own pre-round-33 form (which only recognized a bare `(` as a depth- opening boundary) neither isolated a process substitution's own content NOR correctly paired its matching, bare `)` close -- the unpaired close unconditionally decremented the tracked depth, corrupting isolation tracking for a GENUINELY enclosing real subshell around it too. Independently reproduced live via `classify()` and real bash execution with stand-in `uv`/`gh` binaries on PATH: `TOOL=uv; VERB=harmless; VERB=$(echo install); cat <(VERB=safe) >/dev/null; $TOOL $VERB foo` resolved to `deny=False` even though real bash genuinely runs `uv install foo` (captured argv: "install foo"); the depth-corruption variant, wrapping a real `(...)` subshell around a process substitution followed by a genuine top-level-looking assignment, reproduces identically (confirmed live the outer subshell genuinely still isolates the assignment in real bash: `VERB=install; (cat <(true); VERB=safe); echo "VERB is now: $VERB"` prints `VERB is now: install`). `declare`/`typeset` used INSIDE a function body implicitly localize a variable exactly like `local` does (confirmed live: `VERB=harmless; f() { declare VERB=safe; }; f; echo "VERB=$VERB"` prints `VERB=harmless`, unchanged) -- but the pre-round-33 isolation check recognized only the literal `local` keyword. `TOOL=uv; VERB=harmless; VERB=$(echo install); f() { declare VERB=safe; }; f; $TOOL $VERB foo` resolved to `deny=False` even though real bash genuinely runs `uv install foo`; the `typeset` form and both `_rule_gh_api_write` counterparts reproduce identically. Confirmed through the real wrapper: all six new scenarios (three constructs x two consumers) now deny with exit 2 where they previously allowed with exit 0. Closed by adding `_SUBSHELL_OPEN_TOKENS = frozenset({"(", "<(", ">("})` so `_raw_segments_with_boundaries` treats every process-substitution opener as a depth-opening boundary exactly like a bare `(` (the closing side needed no change -- `<(`/`>(` each already pair with an ordinary bare `)`), and by extending `_segment_tokens_with_scope_isolation`'s own keyword check from `"local" in seg` to also match `"declare"`/ `"typeset"`, treated identically to `local` (always isolating, regardless of whether the segment is genuinely inside a function body or not -- `declare`/`typeset` are ambiguous outside a function, where real bash treats them as an ordinary global assignment, but this file does not otherwise track function-body boundaries, so the safe, conservative default is chosen, accepting a narrow false-positive class rather than reopening a bypass). Regression tests added at every established layer: unit tests for `_raw_segments_with_boundaries`'s new process-substitution handling (including the depth-corruption case) and `_segment_tokens_with_scope_ isolation`'s new `declare`/`typeset` handling, `@given` Hypothesis property tests exercising both functions directly, unit and `@given` tests for both consumer functions' new isolation classes, `classify()`- level end-to-end pins for every reported bypass shape plus false- positive controls, and wrapper-level end-to-end pins against real stand-in `uv`/`gh` binaries (hooks/test_gitapex_check_bash_safety.py, tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI's own whole-codebase invocation), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-33 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 128 +++++++++++-- hooks/test_gitapex_check_bash_safety.py | 57 ++++++ ...st_gitapex_check_bash_safety_properties.py | 173 ++++++++++++++++++ 3 files changed, 344 insertions(+), 14 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 944338d4..ded8ce8a 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4073,6 +4073,18 @@ def _names_reassigned_by_untracked_construct(tokens: list[str]) -> set[str]: return names +_SUBSHELL_OPEN_TOKENS = frozenset({"(", "<(", ">("}) +"""Every token that opens a `(...)`-nesting level `_raw_segments_with_ +boundaries` must track: a bare `(` (subshell grouping/function- +definition parameter list) AND bash's own process-substitution openers +`<(`/`>(`, which this file's own tokenizer fuses into a single token +distinct from a bare `(` (confirmed directly: `tokenize("cat <(VERB= +safe)")` produces `[..., "<(", "VERB=safe", ")", ...]` -- one token for +`<(`, matching a bare `)` for the close, never a bare `(` on its own). +Added by round 33 (issue #1375) -- see `_raw_segments_with_boundaries`'s +own docstring for the live bypass this closes.""" + + def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], int, str | None]]: """Every segment TOKENS splits into (including EMPTY ones, unlike `segment_tokens`'s own filtered output -- an empty segment, e.g. @@ -4085,13 +4097,45 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in of TOKENS with no terminator). Building block for `_segment_tokens_ with_scope_isolation`'s own pipe/background-job detection, which needs to know each segment's own NEIGHBORING boundary tokens, not - just its depth.""" + just its depth. + + CRITICAL bypass found by independent adversarial review (round 33, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: this function's own pre-round-33 form only + recognized a BARE `(` as a depth-opening boundary -- but bash's + process-substitution openers `<(`/`>(` tokenize as their OWN fused, + distinct tokens (`"<("`/`">("`, never a bare `"("`), so a process + substitution's own content was swept into whatever segment was + already active (never isolated) -- and WORSE, its matching, bare `)` + close token WAS recognized and unconditionally decremented the + tracked depth, corrupting tracking for a GENUINELY enclosing `(...)` + subshell around it, prematurely treating tokens still lexically + inside that real subshell as top-level again. `TOOL=uv; + VERB=harmless; VERB=$(echo install); cat <(VERB=safe) >/dev/null; + $TOOL $VERB foo` resolved to `deny=False` even though real bash + genuinely runs `uv install foo`, NOT `safe foo` (confirmed live via + a stand-in `uv` binary on PATH: captured argv `uv called with: + install foo`) -- process substitution runs its own content in a + separate subshell, exactly like `$(...)` command substitution, so it + never leaks back to the parent. The depth-corruption variant, + `TOOL=uv; VERB=harmless; VERB=$(echo install); (cat <(true); + VERB=safe); $TOOL $VERB foo`, reproduces identically (confirmed live + that the OUTER `(...)` genuinely still isolates `VERB=safe` in real + bash: `VERB=install; (cat <(true); VERB=safe); echo "VERB is now: + $VERB"` prints `VERB is now: install`) -- but the pre-fix code + tracked the `VERB=safe` segment at depth 0, having already been + dropped back down by the process substitution's own phantom close. + Closed by recognizing every token in `_SUBSHELL_OPEN_TOKENS` (not + just a bare `(`) as a depth-opening boundary -- `<(`/`>(` each still + pair with an ordinary bare `)` close exactly like a bare `(` does, + so no change to the closing side was needed.""" segments: list[list[str]] = [[]] seg_depths: list[int] = [0] terminators: list[str | None] = [] depth = 0 for tok in tokens: - if tok == "(": + if tok in _SUBSHELL_OPEN_TOKENS: terminators.append(tok) depth += 1 segments.append([]) @@ -4131,16 +4175,27 @@ def _segment_tokens_with_scope_isolation(tokens: list[str]) -> list[tuple[list[s `&` forks into its own subshell; the segment AFTER `&` runs in the parent shell as normal and is NOT isolated by that `&` alone -- only what precedes it is); - - it contains the literal `local` keyword as one of its own tokens - anywhere, not necessarily first -- a function body's own opening - `{` (not itself a `segment_tokens` boundary, so it shares the - SAME segment as the `local` statement that follows it, e.g. - `["{", "local", "VERB=safe"]` for `f() { local VERB=safe; }`) - would otherwise sit at position 0 and hide the keyword from a - first-token-only check (bash's own function-local variable - scoping: a `local NAME=value` inside a function body never leaks - to the function's caller, unlike a plain `NAME=value` in the same - position, which WOULD leak). + - it contains the literal `local`, `declare`, or `typeset` keyword + as one of its own tokens anywhere, not necessarily first -- a + function body's own opening `{` (not itself a `segment_tokens` + boundary, so it shares the SAME segment as the statement that + follows it, e.g. `["{", "local", "VERB=safe"]` for `f() { local + VERB=safe; }`) would otherwise sit at position 0 and hide the + keyword from a first-token-only check. `local` is ALWAYS function- + scoped in real bash (using it outside a function is invalid), so + treating it as isolating is never wrong; `declare`/`typeset` are + genuinely AMBIGUOUS -- they localize a variable ONLY when used + INSIDE a function, and behave as an ordinary, non-isolating global + assignment at the top level of a script, a distinction this + function does not attempt to track (doing so would require + knowing whether the CURRENT segment sits inside a function body's + own `{...}`/`NAME() ...` definition, a form of tracking this file + does not otherwise perform) -- so both are conservatively treated + as ALWAYS isolating, matching this file's own established + deny-on-uncertainty posture; the accepted cost is a narrow false- + positive class (an ordinary, harmless top-level `declare NAME= + value`/`typeset NAME=value` clearing assignment stays + conservatively un-cleared), never a bypass. CRITICAL bypasses found by independent adversarial review (round 32, issue #1375) and independently reproduced live, both via @@ -4213,14 +4268,59 @@ def _segment_tokens_with_scope_isolation(tokens: list[str]) -> list[tuple[list[s this file has followed in every prior round; closing it soundly would require the upstream tokenizer to preserve the presence/ absence of a space between adjacent parens, which is out of this - round's scope.""" + round's scope. + + TWO further gaps found by independent adversarial review (round 33, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: + + Process substitution (`<(cmd)`/`>(cmd)`) also runs its own content + in a separate, isolated subshell, exactly like `$(...)` command + substitution -- but this file's own tokenizer fuses `<(`/`>(` into + their OWN distinct tokens, never a bare `(`, so the pre-round-33 + `_raw_segments_with_boundaries` (which only recognized a bare `(`) + neither isolated a process substitution's own content NOR correctly + paired its matching, bare `)` close, which prematurely decremented + the tracked depth and corrupted isolation tracking for a genuinely + enclosing REAL subshell around it. `TOOL=uv; VERB=harmless; + VERB=$(echo install); cat <(VERB=safe) >/dev/null; $TOOL $VERB foo` + resolved to `deny=False` even though real bash genuinely runs `uv + install foo` (confirmed live via a stand-in `uv` binary on PATH: + captured argv `uv called with: install foo`). Closed by + `_raw_segments_with_boundaries`'s own `_SUBSHELL_OPEN_TOKENS` fix -- + see that function's own docstring for the full live-verified + bypass, including the depth-corruption variant. + + `declare`/`typeset` used INSIDE a function body implicitly localize + a variable exactly like `local` does (confirmed live: `VERB= + harmless; f() { declare VERB=safe; }; f; echo "VERB=$VERB"` prints + `VERB=harmless`, unchanged) -- but the pre-round-33 isolation check + recognized only the literal `local` keyword. `TOOL=uv; VERB= + harmless; VERB=$(echo install); f() { declare VERB=safe; }; f; + $TOOL $VERB foo` resolved to `deny=False` even though real bash + genuinely runs `uv install foo`, NOT `safe foo` (confirmed live via + a stand-in `uv` binary on PATH: captured argv `uv called with: + install foo`); the `typeset` form reproduces identically. Both + reproduce identically for `_rule_gh_api_write` too. Closed by also + checking for `declare`/`typeset` anywhere in the segment, treated + identically to `local` -- see this function's own docstring bullet + above for why both are conservatively treated as ALWAYS isolating + even though real bash only localizes them inside a function body.""" raw = _raw_segments_with_boundaries(tokens) result: list[tuple[list[str], bool]] = [] for i, (seg, depth, terminator) in enumerate(raw): if not seg: continue preceding = raw[i - 1][2] if i > 0 else None - isolated = depth > 0 or terminator in ("|", "&") or preceding == "|" or "local" in seg + isolated = ( + depth > 0 + or terminator in ("|", "&") + or preceding == "|" + or "local" in seg + or "declare" in seg + or "typeset" in seg + ) result.append((seg, isolated)) return result diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index f0555c97..e5374aad 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -433,6 +433,26 @@ def assert_allowed(command: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); true | echo hi; VERB=safe; $TOOL $VERB foo", "real-top-level-static-clear-after-a-harmless-pipeline-stays-allowed", ), + # False-positive guards for the thirty-third-round process- + # substitution and declare/typeset fixes (issue #1375): an ordinary, + # unrelated process substitution elsewhere in the command, or a + # top-level `declare` (never inside a function, so real bash treats + # it as an ordinary global assignment) on a name that was never + # poisoned, must not spuriously deny -- and a harmless process + # substitution earlier in the command must not block a LATER, + # genuine top-level static reassignment from clearing normally. + ( + "TOOL=uv; VERB=safe; cat <(echo hi) >/dev/null; $TOOL $VERB foo", + "unrelated-harmless-process-substitution-alongside-a-never-poisoned-name-stays-allowed", + ), + ( + "TOOL=uv; declare VERB=safe; $TOOL $VERB foo", + "top-level-declare-that-was-never-poisoned-stays-allowed", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); cat <(echo hi) >/dev/null; VERB=safe; $TOOL $VERB foo", + "real-top-level-static-clear-after-a-harmless-process-substitution-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1212,6 +1232,43 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); ( (VERB=totallysafe) ); $TOOL $VERB foo", "deliberately-spaced-double-subshell-distractor-stays-denied", ), + # Found live by Step 8 independent review, thirty-third round (issue + # #1375): process substitution (`<(...)`/`>(...)`) runs its own + # content in a separate, isolated subshell, exactly like `$(...)` + # command substitution, but this classifier's own tokenizer fuses + # `<(`/`>(` into their own distinct tokens, never a bare `(` -- the + # pre-round-33 code neither isolated its content nor correctly + # paired its matching close, corrupting depth tracking for a + # genuinely enclosing real subshell too. `declare`/`typeset` used + # INSIDE a function body also implicitly localize a variable exactly + # like `local` does, which the pre-round-33 code had no concept of. + # Confirmed live via a stand-in `uv`/`gh` binary on PATH that each + # genuinely runs the dangerous command, NOT the process-substitution/ + # declare-scoped distractor value. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); cat <(VERB=safe) >/dev/null; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-process-substitution", + ), + ( + "M=safe; M=$(echo POST); cat <(M=GET) >/dev/null; gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-process-substitution", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); (cat <(true); VERB=safe); $TOOL $VERB foo", + "process-substitution-does-not-corrupt-an-enclosing-subshells-depth", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); f() { declare VERB=safe; }; f; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-declare-declaration", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); f() { typeset VERB=safe; }; f; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-typeset-declaration", + ), + ( + "M=safe; M=$(echo POST); f() { declare M=GET; }; f; gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-declare-declaration", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 5d83d997..30cc3235 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2680,6 +2680,56 @@ def test_segment_tokens_with_scope_isolation_matches_model_for_a_pipe_stage(name assert result == [([f"{name}=first"], False), (["true"], True), ([f"{name}={value}"], True)] +def test_raw_segments_with_boundaries_treats_process_substitution_open_as_a_depth_boundary() -> None: + """CRITICAL bypass regression pin (round-33 independent review, issue + #1375): `<(`/`>(` tokenize as their OWN fused tokens, never a bare + `(` -- `_raw_segments_with_boundaries` must recognize them as + depth-opening boundaries too, or a process substitution's own + content is swept into whatever segment was already active.""" + tokens = ["cat", "<(", "VERB=safe", ")"] + result = checker._raw_segments_with_boundaries(tokens) + assert result == [(["cat"], 0, "<("), (["VERB=safe"], 1, ")"), ([], 0, None)] + + +def test_raw_segments_with_boundaries_process_substitution_does_not_corrupt_an_enclosing_subshells_depth() -> None: + """A process substitution's own matching `)` must decrement depth by + exactly the amount its own `<(`/`>(` incremented it -- never + prematurely closing a GENUINELY enclosing `(...)` subshell around + it.""" + tokens = ["(", "cat", "<(", "true", ")", ";", "VERB=safe", ")"] + result = checker._raw_segments_with_boundaries(tokens) + assert result[0] == ([], 0, "(") + assert result[1] == (["cat"], 1, "<(") + assert result[2] == (["true"], 2, ")") + assert result[3] == ([], 1, ";") + assert result[4] == (["VERB=safe"], 1, ")") + assert result[5] == ([], 0, None) + + +def test_segment_tokens_with_scope_isolation_marks_a_declare_declaration_isolated() -> None: + tokens = ["{", "declare", "VERB=safe"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["{", "declare", "VERB=safe"], True)] + + +def test_segment_tokens_with_scope_isolation_marks_a_typeset_declaration_isolated() -> None: + tokens = ["{", "typeset", "VERB=safe"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["{", "typeset", "VERB=safe"], True)] + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_VALUES) +def test_segment_tokens_with_scope_isolation_matches_model_for_a_process_substitution(name: str, value: str) -> None: + """Model-based, exercising `_segment_tokens_with_scope_isolation` + directly (issue #1178's own detection-logic property-coverage + requirement): for ANY identifier assigned a value inside a `<(...)` + process substitution, that segment is marked isolated.""" + tokens = ["cat", "<(", f"{name}={value}", ")"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["cat"], False), ([f"{name}={value}"], True)] + + def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_subshell_clear() -> None: """CRITICAL bypass regression pin (round-31 independent review, issue #1375): a static reassignment written INSIDE a `(...)` subshell @@ -2761,6 +2811,24 @@ def test_names_reassigned_from_a_static_value_matches_model_for_a_pipe_stage_cle assert name in checker._names_reassigned_from_a_static_value(tokens) +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_process_substitution_clear() -> None: + """CRITICAL bypass regression pin (round-33 independent review, issue + #1375): a process substitution runs its own content in a separate, + isolated subshell, exactly like `$(...)` command substitution -- a + static reassignment inside `<(...)` can never reach the parent + shell's own copy of the name.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "cat", "<(", "VERB=safe", ")"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_declare_clear() -> None: + """Same round, the `declare`-inside-a-function counterpart: bash + localizes `declare`/`typeset` inside a function body exactly like + `local`.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "{", "declare", "VERB=safe", "}"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, final_static_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model_for_a_static_dynamic_static_sequence( @@ -2949,6 +3017,20 @@ def test_names_cleared_by_a_later_static_reassignment_matches_model_for_a_pipe_s assert name not in checker._names_cleared_by_a_later_static_reassignment(tokens, {name}) +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_process_substitution() -> None: + """CRITICAL bypass regression pin (round-33 independent review, issue + #1375): a static reassignment inside a `<(...)` process substitution + never reaches the parent shell's own copy of the name -- must not + clear an append-poisoned candidate.""" + tokens = ["VERB=inst", "VERB+=all", ";", "cat", "<(", "VERB=safe", ")"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_declare_declaration() -> None: + tokens = ["VERB=inst", "VERB+=all", ";", "{", "declare", "VERB=safe", "}"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + def test_names_cleared_by_a_later_static_reassignment_clears_after_a_printf_v() -> None: tokens = ["printf", "-v", "VERB", "%s", "install", ";", "VERB=safe"] assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} @@ -3370,6 +3452,97 @@ def test_classify_denies_a_deliberately_spaced_double_subshell_distractor() -> N assert verdict.deny is True +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_process_substitution_clear() -> None: + """CRITICAL bypass regression pin (round-33 independent review, + issue #1375): a process substitution runs its own content in a + separate, isolated subshell, exactly like `$(...)` command + substitution -- but this classifier's own tokenizer fuses `<(`/`>(` + into their own distinct tokens, never a bare `(`, so the pre-round- + 33 code neither isolated its content nor correctly paired its + matching close, corrupting depth tracking too. Confirmed live via a + stand-in `uv` binary on PATH that this genuinely runs `uv install + foo`, NOT `safe foo`.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); cat <(VERB=safe) >/dev/null; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_process_substitution_clear() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`. + Confirmed live via a stand-in `gh` binary on PATH that this + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST`, a genuine + unreviewed write.""" + verdict = checker.classify("M=safe; M=$(echo POST); cat <(M=GET) >/dev/null; gh api repos/o/r/pulls/1/merge -X $M") + assert verdict.deny is True + + +def test_classify_denies_a_process_substitution_that_corrupts_an_enclosing_subshells_depth() -> None: + """The depth-corruption variant: a process substitution's own + phantom close must not prematurely decrement tracked depth for a + GENUINELY enclosing `(...)` subshell around it. Confirmed live that + the outer subshell genuinely still isolates `VERB=safe` in real bash + (`VERB=install; (cat <(true); VERB=safe); echo "VERB is now: $VERB"` + prints `VERB is now: install`).""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); (cat <(true); VERB=safe); $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_declare_declaration_clear() -> None: + """Same round, the `declare`-inside-a-function counterpart: bash + localizes `declare`/`typeset` inside a function body exactly like + `local`. Confirmed live via a stand-in `uv` binary on PATH that this + genuinely runs `uv install foo`, NOT `safe foo`.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); f() { declare VERB=safe; }; f; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_typeset_declaration_clear() -> None: + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); f() { typeset VERB=safe; }; f; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_declare_declaration_clear() -> None: + verdict = checker.classify( + "M=safe; M=$(echo POST); f() { declare M=GET; }; f; gh api repos/o/r/pulls/1/merge -X $M" + ) + assert verdict.deny is True + + +def test_classify_allows_an_unrelated_harmless_process_substitution_alongside_a_never_poisoned_name() -> None: + """No over-correction: an ordinary, unrelated process substitution + elsewhere in the command must not spuriously deny a command whose + watched name was never poisoned at all.""" + verdict = checker.classify("TOOL=uv; VERB=safe; cat <(echo hi) >/dev/null; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_top_level_declare_that_was_never_poisoned() -> None: + """No over-correction: `declare`/`typeset` at the TOP LEVEL of a + script (not inside any function) behaves as an ordinary global + assignment in real bash -- a name that was never poisoned to begin + with must still be allowed even though `declare` is conservatively + treated as always-isolating for the CLEARING check specifically.""" + verdict = checker.classify("TOOL=uv; declare VERB=safe; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_process_substitution() -> None: + """No over-correction: a harmless process substitution earlier in + the command must not block a LATER, genuine top-level static + reassignment from clearing poisoning normally.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); cat <(echo hi) >/dev/null; VERB=safe; $TOOL $VERB foo" + ) + assert verdict.deny is False + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 92da11c192faf3102de35ceb6c593614017bc29f Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 02:33:00 +0000 Subject: [PATCH 39/46] fix(hooks): recognize coproc and $"..."-fused local/declare/typeset as scope-isolating A fresh, independent adversarial review of this PR's current head (round 34) found two more sibling gaps in the same scope-isolation area rounds 31-33 already closed for `(...)` subshells, pipe stages, background jobs, `local`/`declare`/`typeset` declarations, and process substitution. `coproc { ... }` (bash's own coprocess syntax) forks its body to run asynchronously in a subshell connected by a pipe, exactly like `cmd &`, with no non-isolating usage at all -- but the pre-round-34 isolation check had no concept of `coproc` whatsoever. Independently reproduced live via `classify()` and real bash execution with stand-in `uv`/`gh` binaries on PATH: `TOOL=uv; VERB=harmless; VERB=$(echo install); coproc { VERB=safe; }; wait; $TOOL $VERB foo` resolved to `deny=False` even though real bash genuinely runs `uv install foo` (captured argv: "install foo"); the `_rule_gh_api_write` counterpart reproduces identically. Bash's `$"..."` locale-translated-string syntax fuses the `$` prefix onto the dequoted string content -- this classifier's own tokenizer turns `$"local"` into the single token `$local`, never a bare `local`, so the exact-membership check the round-32/33 fixes relied on (`"local" in seg`) never matched it, even though `$"local"` genuinely invokes the `local` builtin in real bash when used in command-starting position (confirmed live: `f() { $"local" VERB=safe; }; f` leaves the caller's own `$VERB` untouched, exactly like a bare `local VERB=safe` would). `TOOL=uv; VERB=harmless; VERB=$(echo install); f() { $"local" VERB=safe; }; f; $TOOL $VERB foo` resolved to `deny=False` even though real bash genuinely runs `uv install foo`. Confirmed through the real wrapper: all four new scenarios (two constructs x two consumers) now deny with exit 2 where they previously allowed with exit 0. Closed by extracting a shared `_SCOPE_LOCALIZING_KEYWORDS` set (now `local`, `declare`, `typeset`, `coproc`) and a new `_seg_has_a_scope_ localizing_keyword` helper that checks each token BOTH as-is and with a single leading `$` stripped, catching the `$"..."`-fused form for all four keywords uniformly -- `coproc` is treated identically to `local` (always isolating, no ambiguous non-isolating usage exists in real bash). Regression tests added at every established layer: unit tests for `_seg_has_a_scope_localizing_keyword`/`_segment_tokens_with_scope_ isolation`'s new `coproc`/`$"..."`-fused handling directly, a `@given` Hypothesis property test exercising the new helper by name (issue #1178's own coverage requirement), unit and `classify()`-level tests for both consumer functions' new isolation classes plus false-positive controls (an unrelated coproc, an unrelated plain `$local` variable reference in a different segment, a real top-level clear after a harmless coproc), and wrapper-level end-to-end pins against real stand-in `uv`/`gh` binaries (hooks/test_gitapex_check_bash_safety.py, tests/test_gitapex_check_bash_safety_properties.py). Full gate suite green: ruff check/format, mypy, xenon (CI's own whole-codebase invocation), the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on hooks/gitapex_check_bash_safety.py (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-34 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 95 ++++++++++--- hooks/test_gitapex_check_bash_safety.py | 47 +++++++ ...st_gitapex_check_bash_safety_properties.py | 133 ++++++++++++++++++ 3 files changed, 257 insertions(+), 18 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index ded8ce8a..a16f7e2e 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4155,6 +4155,33 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in return list(zip(segments, seg_depths, terminators, strict=True)) +_SCOPE_LOCALIZING_KEYWORDS = frozenset({"local", "declare", "typeset", "coproc"}) +"""Every bash keyword `_segment_tokens_with_scope_isolation` treats as +ALWAYS scope-isolating wherever it appears in a segment (see that +function's own docstring for why each is safe to treat this way even +though `declare`/`typeset` are genuinely ambiguous outside a function). +Added by round 32 (`local`/`declare`/`typeset`) and round 34 (`coproc`), +issue #1375.""" + + +def _seg_has_a_scope_localizing_keyword(seg: list[str]) -> bool: + """True when SEG contains any of `_SCOPE_LOCALIZING_KEYWORDS` as one + of its own tokens, checking each token BOTH as-is and with a single + leading `$` stripped -- bash's own `$"..."` locale-translated-string + syntax fuses the `$` prefix onto the dequoted string content (e.g. + `$"local"` tokenizes as the single token `$local`, never a bare + `local`), so an exact, unstripped membership check misses it even + though it genuinely invokes the keyword in real bash when used in + command-starting position. Added by round 34 (issue #1375) -- see + `_segment_tokens_with_scope_isolation`'s own docstring for the live + bypass this closes.""" + for tok in seg: + bare = tok[1:] if tok.startswith("$") else tok + if bare in _SCOPE_LOCALIZING_KEYWORDS: + return True + return False + + def _segment_tokens_with_scope_isolation(tokens: list[str]) -> list[tuple[list[str], bool]]: """Like `segment_tokens`, but pairs each returned (non-empty) segment with whether it is ISOLATED from the parent shell's own @@ -4175,15 +4202,22 @@ def _segment_tokens_with_scope_isolation(tokens: list[str]) -> list[tuple[list[s `&` forks into its own subshell; the segment AFTER `&` runs in the parent shell as normal and is NOT isolated by that `&` alone -- only what precedes it is); - - it contains the literal `local`, `declare`, or `typeset` keyword - as one of its own tokens anywhere, not necessarily first -- a - function body's own opening `{` (not itself a `segment_tokens` - boundary, so it shares the SAME segment as the statement that - follows it, e.g. `["{", "local", "VERB=safe"]` for `f() { local - VERB=safe; }`) would otherwise sit at position 0 and hide the - keyword from a first-token-only check. `local` is ALWAYS function- - scoped in real bash (using it outside a function is invalid), so - treating it as isolating is never wrong; `declare`/`typeset` are + - it contains the literal `local`, `declare`, `typeset`, or `coproc` + keyword as one of its own tokens anywhere, not necessarily first + -- a function body's own opening `{` (not itself a `segment_ + tokens` boundary, so it shares the SAME segment as the statement + that follows it, e.g. `["{", "local", "VERB=safe"]` for `f() { + local VERB=safe; }`) would otherwise sit at position 0 and hide + the keyword from a first-token-only check; a token is also + checked with a single leading `$` stripped (see `_SCOPE_ + LOCALIZING_KEYWORDS`'s own docstring for why -- bash's `$"..."` + locale-string syntax fuses onto the dequoted keyword text, e.g. + `$"local"` tokenizes as `$local`, never a bare `local`). `local` + and `coproc` are ALWAYS isolating in real bash (`local` outside a + function is invalid; `coproc`'s own body always runs + asynchronously in a forked subshell connected by a pipe, exactly + like `cmd &`, with no non-isolating usage at all), so treating + either as isolating is never wrong; `declare`/`typeset` are genuinely AMBIGUOUS -- they localize a variable ONLY when used INSIDE a function, and behave as an ordinary, non-isolating global assignment at the top level of a script, a distinction this @@ -4306,21 +4340,46 @@ def _segment_tokens_with_scope_isolation(tokens: list[str]) -> list[tuple[list[s checking for `declare`/`typeset` anywhere in the segment, treated identically to `local` -- see this function's own docstring bullet above for why both are conservatively treated as ALWAYS isolating - even though real bash only localizes them inside a function body.""" + even though real bash only localizes them inside a function body. + + TWO further gaps found by independent adversarial review (round 34, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: + + `coproc { ... }` (bash's own coprocess syntax) forks its body to run + asynchronously in a subshell connected by a pipe, exactly like + `cmd &`, with no non-isolating usage at all -- but the pre-round-34 + isolation check had no concept of `coproc` whatsoever. `TOOL=uv; + VERB=harmless; VERB=$(echo install); coproc { VERB=safe; }; wait; + $TOOL $VERB foo` resolved to `deny=False` even though real bash + genuinely runs `uv install foo` (confirmed live via a stand-in `uv` + binary on PATH: captured argv `uv called with: install foo`). + Closed by also checking for `coproc` anywhere in the segment, + treated the same as `local` (always isolating, no ambiguous usage). + + Bash's `$"..."` locale-translated-string syntax fuses the `$` prefix + onto the dequoted string content -- `$"local"` tokenizes as a SINGLE + token `$local`, never a bare `local`, so the exact-membership check + above (`"local" in seg`) never matched it, even though `$"local"` + genuinely invokes the `local` builtin in real bash when used in + command-starting position (confirmed live: `f() { $"local" VERB= + safe; }; f` leaves the caller's own `$VERB` untouched, exactly like + a bare `local VERB=safe` would). `TOOL=uv; VERB=harmless; VERB= + $(echo install); f() { $"local" VERB=safe; }; f; $TOOL $VERB foo` + resolved to `deny=False` even though real bash genuinely runs `uv + install foo` (confirmed live via a stand-in `uv` binary on PATH: + captured argv `uv called with: install foo`). Closed by checking + each token with a single leading `$` stripped too (via + `_SCOPE_LOCALIZING_KEYWORDS`/`_seg_has_a_scope_localizing_keyword`), + catching the fused `$"..."` form for all four keywords uniformly.""" raw = _raw_segments_with_boundaries(tokens) result: list[tuple[list[str], bool]] = [] for i, (seg, depth, terminator) in enumerate(raw): if not seg: continue preceding = raw[i - 1][2] if i > 0 else None - isolated = ( - depth > 0 - or terminator in ("|", "&") - or preceding == "|" - or "local" in seg - or "declare" in seg - or "typeset" in seg - ) + isolated = depth > 0 or terminator in ("|", "&") or preceding == "|" or _seg_has_a_scope_localizing_keyword(seg) result.append((seg, isolated)) return result diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index e5374aad..6be4f7dd 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -453,6 +453,26 @@ def assert_allowed(command: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); cat <(echo hi) >/dev/null; VERB=safe; $TOOL $VERB foo", "real-top-level-static-clear-after-a-harmless-process-substitution-stays-allowed", ), + # False-positive guards for the thirty-fourth-round coproc and + # `$"local"` fixes (issue #1375): an ordinary, unrelated coproc + # elsewhere in the command must not spuriously deny a command whose + # watched name was never poisoned at all, an ordinary `$local` + # variable reference (not the `$"local"` locale-string form) in a + # DIFFERENT segment must not either, and a harmless coproc earlier in + # the command must not block a LATER, genuine top-level static + # reassignment from clearing normally. + ( + "TOOL=uv; VERB=safe; coproc { echo hi; }; wait; $TOOL $VERB foo", + "unrelated-harmless-coproc-alongside-a-never-poisoned-name-stays-allowed", + ), + ( + "TOOL=uv; echo $local; VERB=safe; $TOOL $VERB foo", + "unrelated-plain-dollar-local-variable-reference-stays-allowed", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); coproc { echo hi; }; wait; VERB=safe; $TOOL $VERB foo", + "real-top-level-static-clear-after-a-harmless-coproc-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1269,6 +1289,33 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "M=safe; M=$(echo POST); f() { declare M=GET; }; f; gh api repos/o/r/pulls/1/merge -X $M", "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-declare-declaration", ), + # Found live by Step 8 independent review, thirty-fourth round (issue + # #1375): `coproc { ... }` forks its body to run asynchronously in a + # subshell connected by a pipe, exactly like `cmd &`, with no + # non-isolating usage at all -- the pre-round-34 isolation check had + # no concept of `coproc` whatsoever. Bash's `$"..."` locale- + # translated-string syntax also fuses the `$` prefix onto the + # dequoted string content -- `$"local"` tokenizes as a single token + # `$local`, never a bare `local`, but genuinely invokes the `local` + # builtin in command-starting position. Confirmed live via a stand-in + # `uv`/`gh` binary on PATH that each genuinely runs the dangerous + # command, NOT the coproc/`$"local"`-scoped distractor value. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); coproc { VERB=safe; }; wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-coproc", + ), + ( + "M=safe; M=$(echo POST); coproc { M=GET; }; wait; gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-coproc", + ), + ( + 'TOOL=uv; VERB=harmless; VERB=$(echo install); f() { $"local" VERB=safe; }; f; $TOOL $VERB foo', + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-dollar-quoted-local", + ), + ( + 'M=safe; M=$(echo POST); f() { $"local" M=GET; }; f; gh api repos/o/r/pulls/1/merge -X $M', + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-dollar-quoted-local", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 30cc3235..a803038e 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2718,6 +2718,34 @@ def test_segment_tokens_with_scope_isolation_marks_a_typeset_declaration_isolate assert result == [(["{", "typeset", "VERB=safe"], True)] +def test_segment_tokens_with_scope_isolation_marks_a_coproc_body_isolated() -> None: + """CRITICAL bypass regression pin (round-34 independent review, issue + #1375): `coproc { ... }` forks its body to run asynchronously in a + subshell connected by a pipe, exactly like `cmd &`, with no + non-isolating usage at all.""" + tokens = ["coproc", "{", "VERB=safe"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["coproc", "{", "VERB=safe"], True)] + + +def test_segment_tokens_with_scope_isolation_marks_a_dollar_quoted_local_isolated() -> None: + """CRITICAL bypass regression pin (round-34 independent review, issue + #1375): bash's `$"..."` locale-translated-string syntax fuses the + `$` prefix onto the dequoted string content -- `$"local"` tokenizes + as the single token `$local`, never a bare `local` -- but genuinely + invokes the `local` builtin in command-starting position.""" + tokens = ["{", "$local", "VERB=safe"] + result = checker._segment_tokens_with_scope_isolation(tokens) + assert result == [(["{", "$local", "VERB=safe"], True)] + + +def test_seg_has_a_scope_localizing_keyword_ignores_an_unrelated_dollar_reference() -> None: + """No false positive: an ordinary `$NAME` reference to a variable + that is NOT one of the localizing keywords must not match, with or + without the leading `$` stripped.""" + assert checker._seg_has_a_scope_localizing_keyword(["echo", "$localvar"]) is False + + @_PROPERTIES @given(name=_IDENTIFIERS, value=_VALUES) def test_segment_tokens_with_scope_isolation_matches_model_for_a_process_substitution(name: str, value: str) -> None: @@ -2730,6 +2758,19 @@ def test_segment_tokens_with_scope_isolation_matches_model_for_a_process_substit assert result == [(["cat"], False), ([f"{name}={value}"], True)] +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_VALUES) +def test_seg_has_a_scope_localizing_keyword_matches_model_for_a_dollar_quoted_keyword(name: str, value: str) -> None: + """Model-based, exercising `_seg_has_a_scope_localizing_keyword` + directly (issue #1178's own detection-logic property-coverage + requirement): for ANY identifier, a segment containing the `$"..."`- + fused form of any of the four localizing keywords is recognized -- + and a segment with no such keyword, fused or bare, is not.""" + for keyword in ("local", "declare", "typeset", "coproc"): + assert checker._seg_has_a_scope_localizing_keyword(["{", f"${keyword}", f"{name}={value}"]) is True + assert checker._seg_has_a_scope_localizing_keyword([f"{name}={value}"]) is False + + def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_subshell_clear() -> None: """CRITICAL bypass regression pin (round-31 independent review, issue #1375): a static reassignment written INSIDE a `(...)` subshell @@ -2829,6 +2870,22 @@ def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_declare_c assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_coproc_clear() -> None: + """CRITICAL bypass regression pin (round-34 independent review, issue + #1375): `coproc { ... }` forks its body into an asynchronous + subshell connected by a pipe, exactly like `cmd &` -- a static + reassignment inside it never reaches the parent shell's own copy of + the name.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "coproc", "{", "VERB=safe", "}"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_dollar_quoted_local_clear() -> None: + """Same round, the `$"local"`-fused-token counterpart.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "{", "$local", "VERB=safe", "}"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, final_static_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model_for_a_static_dynamic_static_sequence( @@ -3031,6 +3088,20 @@ def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_decla assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_coproc() -> None: + """CRITICAL bypass regression pin (round-34 independent review, issue + #1375): a static reassignment inside a `coproc { ... }` body never + reaches the parent shell's own copy of the name -- must not clear an + append-poisoned candidate.""" + tokens = ["VERB=inst", "VERB+=all", ";", "coproc", "{", "VERB=safe", "}"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_dollar_quoted_local() -> None: + tokens = ["VERB=inst", "VERB+=all", ";", "{", "$local", "VERB=safe", "}"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + def test_names_cleared_by_a_later_static_reassignment_clears_after_a_printf_v() -> None: tokens = ["printf", "-v", "VERB", "%s", "install", ";", "VERB=safe"] assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} @@ -3543,6 +3614,68 @@ def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_process_ assert verdict.deny is False +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_coproc_clear() -> None: + """CRITICAL bypass regression pin (round-34 independent review, + issue #1375): `coproc { ... }` forks its body to run asynchronously + in a subshell connected by a pipe, exactly like `cmd &`, with no + non-isolating usage at all. Confirmed live via a stand-in `uv` + binary on PATH that this genuinely runs `uv install foo`, NOT + `safe foo`.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); coproc { VERB=safe; }; wait; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_coproc_clear() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`. + Confirmed live via a stand-in `gh` binary on PATH that this + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST`, a genuine + unreviewed write.""" + verdict = checker.classify("M=safe; M=$(echo POST); coproc { M=GET; }; wait; gh api repos/o/r/pulls/1/merge -X $M") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_dollar_quoted_local_clear() -> None: + """Same round, the `$"local"`-fused-token counterpart: bash's own + locale-translated-string syntax fuses the `$` prefix onto the + dequoted string content, so `$"local"` tokenizes as a single token + `$local`, never a bare `local`, but genuinely invokes the `local` + builtin in command-starting position. Confirmed live via a stand-in + `uv` binary on PATH that this genuinely runs `uv install foo`, NOT + `safe foo`.""" + verdict = checker.classify( + 'TOOL=uv; VERB=harmless; VERB=$(echo install); f() { $"local" VERB=safe; }; f; $TOOL $VERB foo' + ) + assert verdict.deny is True + + +def test_classify_allows_an_unrelated_harmless_coproc_alongside_a_never_poisoned_name() -> None: + """No over-correction: an ordinary, unrelated coproc elsewhere in + the command must not spuriously deny a command whose watched name + was never poisoned at all.""" + verdict = checker.classify("TOOL=uv; VERB=safe; coproc { echo hi; }; wait; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_an_unrelated_plain_dollar_local_variable_reference() -> None: + """No over-correction: an ordinary `$local` variable reference (not + the `$"local"` locale-string form) in a DIFFERENT segment from a + genuine top-level clearing assignment must not spuriously deny.""" + verdict = checker.classify("TOOL=uv; echo $local; VERB=safe; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_coproc() -> None: + """No over-correction: a harmless coproc earlier in the command must + not block a LATER, genuine top-level static reassignment from + clearing poisoning normally.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); coproc { echo hi; }; wait; VERB=safe; $TOOL $VERB foo" + ) + assert verdict.deny is False + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From ac1e8b379e7f806ca54457f3d5f9eada1a475260 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 02:59:22 +0000 Subject: [PATCH 40/46] fix(hooks): recognize piped/backgrounded compound-command groups as scope-isolating A fresh, independent adversarial review of this PR's current head (round 35) found a new sibling gap in the same scope-isolation area rounds 31-34 already closed for `(...)` subshells, pipe stages, background jobs, `local`/`declare`/`typeset`/`coproc` declarations, and process substitution. None of `{`, `}`, `while`, `do`, `done`, `until`, `for`, `select`, `if`, `then`, `fi` were recognized at all -- but a `{...}` brace group or a `while`/`until`/`for`/`select`/`if` compound command, backgrounded or piped AS A WHOLE, forks exactly like a subshell: a fake "the value is safe now" reassignment written inside one can never actually reach the parent shell's own copy of the name. Independently reproduced live via classify() and real bash execution with stand-in uv/gh binaries on PATH: a backgrounded brace group around a static reassignment resolved to deny=False even though real bash genuinely runs the earlier, still- dynamic value's own dangerous command (captured argv confirmed against a stand-in uv binary); the piped-brace-group, piped-while-loop, backgrounded-if, backgrounded-for, doubly-nested-piped-group, and pipe-receiving-group's-later-statement shapes all reproduce identically, as does the _rule_gh_api_write counterpart. Two negative controls confirm a correct fix cannot simply treat every brace/while/until/for/select/if as unconditionally isolating (the same false-positive trade-off already navigated for parenthesized subshells in round 31/32, and deliberately not extended to arithmetic ((...))): a bare compound-command group with no trailing background or pipe boundary genuinely leaks to the parent in real bash and must stay allowed. A third live-verified concern surfaced during this round's own design review, not by the reviewing subagent: an initial stack-based matching design that scanned every token position (not just position 0) for a close keyword desynced against a literal, non-syntactic argument (a bare "fi" as a plain command argument) sitting inside the same group's own segment -- it popped the real group's own stack entry against that spurious match, leaving the REAL closing brace unmatched and the assignment wrongly left unisolated, reopening the very bypass this change exists to close. Closed by requiring a close keyword to sit at position 0 of its own raw segment (grammar-accurate: bash always requires a closing brace/done/fi to start a fresh command), while still counting every leading, contiguous run of open keywords in a segment (bash places no such requirement between an outer and an immediately-nested opener) -- confirmed live post-fix that the literal- argument distractor case still correctly denies. Closed via a new open/close keyword-set pair and a bracket-matching helper that marks every raw segment inside a group isolated exactly when the group's own outer boundary is a pipe stage, is backgrounded, or is itself the receiving side of a pipe -- wired into the existing scope-isolation segmentation alongside the subshell/pipe/background/ local checks already there. Regression tests added at every established layer: unit tests for the new bracket-matching function directly (including the nested-open, pipe-receiving-later-statement, and literal-close-keyword-argument safety cases), a Hypothesis property test exercising it by name (issue #1178's own coverage requirement), unit and classify()-level tests for both consumer functions' new isolation classes plus false-positive controls (an unrelated backgrounded group, a real top-level clear after a harmless one, three bare-group negative controls), and wrapper-level end-to-end pins against real stand-in uv/gh binaries. Full gate suite green: ruff check/format, mypy, the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on the classifier file (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-35 baseline). The one xenon complexity finding surfaced by a whole-codebase run is confirmed byte-identical to origin/main and outside this diff entirely -- a pre-existing condition, not introduced by this round. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 151 +++++++- hooks/test_gitapex_check_bash_safety.py | 70 ++++ ...st_gitapex_check_bash_safety_properties.py | 330 ++++++++++++++++++ 3 files changed, 549 insertions(+), 2 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index a16f7e2e..4271554f 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4182,6 +4182,133 @@ def _seg_has_a_scope_localizing_keyword(seg: list[str]) -> bool: return False +_GROUP_OPEN_KEYWORDS = frozenset({"{", "while", "until", "for", "select", "if"}) +_GROUP_CLOSE_KEYWORDS = frozenset({"}", "done", "fi"}) +"""The compound-command delimiters `_segment_indices_isolated_by_a_ +piped_or_backgrounded_compound_group` bracket-matches. Unlike `_SUBSHELL_ +OPEN_TOKENS`, none of these tokens themselves make their own content +isolated -- a bare `{ VERB=safe; }`/`while ...; done`/`if ...; fi` with no +trailing `&`/`|` genuinely LEAKS its assignments to the parent shell in +real bash (confirmed live, see the round-35 paragraph below) -- isolation +here depends entirely on whether the GROUP AS A WHOLE is itself a +pipeline stage or a backgrounded job, the same test already applied to an +ordinary segment, evaluated once at the group's own matching open/close +pair. Added by round 35 (issue #1375).""" + + +def _segment_indices_isolated_by_a_piped_or_backgrounded_compound_group( + raw: list[tuple[list[str], int, str | None]], +) -> set[int]: + """Every index into RAW (the `_raw_segments_with_boundaries` output) + that lies inside a `{...}`/`while`/`until`/`for`/`select`/`if`... + `done`/`fi` compound-command group whose own OUTER closing boundary + is itself a pipe stage, is immediately followed by `&`, or is itself + the RECEIVING side of a pipe -- bash forks the ENTIRE group as one + unit in each of these cases, so a plain static assignment anywhere + inside it, at any nesting depth, must never be trusted to clear an + earlier poisoning. + + CRITICAL bypass found by independent adversarial review (round 35, + issue #1375) and independently reproduced live, both via `classify()` + and via real bash execution with a stand-in `uv`/`gh` binary on PATH: + rounds 31-34 taught `_raw_segments_with_boundaries`/`_segment_tokens_ + with_scope_isolation` about `(...)` subshells, `<(`/`>(` process + substitution, `|` pipeline stages, `cmd &` backgrounding, and `local`/ + `declare`/`typeset`/`coproc` -- but none of `{`, `}`, `while`, `do`, + `done`, `until`, `for`, `select`, `if`, `then`, `fi` were ever + recognized at all, even though a `{...}` brace group or a `while`/ + `until`/`for`/`select`/`if` compound command, backgrounded or piped + AS A WHOLE, forks exactly like a subshell -- a fake "the value is + safe now" reassignment written inside one can never actually reach + the parent shell's own copy of the name, but the pre-round-35 code + trusted it exactly like an ordinary top-level reassignment. `TOOL=uv; + VERB=harmless; VERB=$(echo install); { VERB=safe; } & wait; $TOOL + $VERB foo` resolved to `deny=False` even though real bash genuinely + runs `uv install foo` (confirmed live via a stand-in `uv` binary on + PATH: captured argv `uv called with: install foo`) -- the brace + group's own `VERB=safe` never reaches the parent shell once the whole + group is backgrounded. `{ VERB=safe; } | cat` (piped instead of + backgrounded), `if true; then VERB=safe; fi & wait`, `for i in 1; do + VERB=safe; done & wait`, a doubly-nested `{ { VERB=safe; }; } | cat`, + and a pipe-RECEIVING group whose assignment is not even the group's + OWN first statement (`echo x | { true; VERB=safe; }; wait`) all + reproduce identically (captured argv in every case: `uv called with: + install foo`, never `safe foo`); `_rule_gh_api_write` reproduces + identically too (e.g. `M=safe; M=$(echo POST); { true; M=GET; } & + wait; gh api repos/o/r/pulls/1/merge -X $M` -- real bash genuinely + runs `-X POST`, a genuine unreviewed write). Confirmed through the + real wrapper: every shape now denies with exit 2 where it previously + allowed with exit 0. + + Two negative controls confirm a correct fix cannot simply treat every + `{`/`while`/`until`/`for`/`select`/`if` as UNCONDITIONALLY isolating + (the same false-positive trade-off already navigated for `(...)` in + round 31/32 and deliberately NOT extended to `((...))` -- see + `_segment_tokens_with_scope_isolation`'s own `((...))` paragraph): a + bare `{ VERB=safe; }`/`if true; then VERB=safe; fi`/`for i in 1; do + VERB=safe; done` with NO trailing `&`/`|` genuinely LEAKS to the + parent in real bash (confirmed live: each variant's stand-in `uv` + call captures `uv called with: safe foo`, not `install foo`) and + `classify()` already agreed even before this fix -- isolation here + is conditioned on the group's own outer boundary, never on the mere + presence of one of these keywords. + + A group is recognized by treating a maximal, CONTIGUOUS run of + `_GROUP_OPEN_KEYWORDS` tokens starting at position 0 of a raw segment + as that many nested opens (bash's own grammar requires each of these + reserved words to start a fresh command, but places no requirement + that a nested group's own opener be separated from an enclosing + group's opener by anything but whitespace -- confirmed live: + `tokenize("{ { echo hi; }; }")` produces adjacent `"{"`, `"{"` tokens + in the SAME raw segment, with no boundary token between them, since + `{` is not itself one of `_raw_segments_with_boundaries`'s own + boundary tokens) -- and any `_GROUP_CLOSE_KEYWORDS` token found ONLY + at position 0 of a raw segment as closing one level. The position-0 + requirement for BOTH is not an arbitrary restriction: it mirrors what + real bash's own grammar already guarantees -- `}`/`done`/`fi` (and + an opener that begins a genuine nested group) must always be the + first word of the command they start, so a literal, non-syntactic + occurrence of one of these words LATER in a segment (e.g. `echo fi`, + where `fi` is merely `echo`'s own argument) can never be its real, + syntactic use. This matters for SAFETY, not just accuracy: an + earlier stack-based design that scanned every token position for a + close (not just position 0) was found, during this round's own + design review, to desync against exactly this kind of literal + argument -- `{ echo fi; VERB=safe; } & wait` would have popped the + real group's own stack entry against the SPURIOUS `fi` inside `echo + fi`'s own segment, leaving the REAL closing `}` unmatched and + `VERB=safe` wrongly left unisolated, reopening the very bypass this + function exists to close. Verified live that this classifier's own + fixed position-0-only close check does NOT reopen that hole: `TOOL= + uv; VERB=harmless; VERB=$(echo install); { echo fi; VERB=safe; } & + wait; $TOOL $VERB foo` denies correctly post-fix (real bash: `uv + called with: install foo`, matching `classify()`'s own `deny=True`). + + A group is isolated when its own outermost closing raw segment's + terminator is `|` or `&` (the group forks because it is itself a + non-final-or-final pipe stage, or because it is backgrounded), OR + when the raw segment immediately preceding the group's own opening + raw segment terminates with `|` (the group is itself the RECEIVING + side of a pipe -- case H above).""" + stack: list[int] = [] + isolated: set[int] = set() + for i, (seg, _depth, terminator) in enumerate(raw): + opens = 0 + for tok in seg: + if tok not in _GROUP_OPEN_KEYWORDS: + break + opens += 1 + for _ in range(opens): + stack.append(i) + if stack and seg and seg[0] in _GROUP_CLOSE_KEYWORDS: + start = stack.pop() + preceding = raw[i - 1][2] if i > 0 else None + preceding_of_open = raw[start - 1][2] if start > 0 else None + if terminator in ("|", "&") or preceding == "|" or preceding_of_open == "|": + isolated.update(range(start, i + 1)) + return isolated + + def _segment_tokens_with_scope_isolation(tokens: list[str]) -> list[tuple[list[str], bool]]: """Like `segment_tokens`, but pairs each returned (non-empty) segment with whether it is ISOLATED from the parent shell's own @@ -4372,14 +4499,34 @@ def _segment_tokens_with_scope_isolation(tokens: list[str]) -> list[tuple[list[s captured argv `uv called with: install foo`). Closed by checking each token with a single leading `$` stripped too (via `_SCOPE_LOCALIZING_KEYWORDS`/`_seg_has_a_scope_localizing_keyword`), - catching the fused `$"..."` form for all four keywords uniformly.""" + catching the fused `$"..."` form for all four keywords uniformly. + + ONE further gap found by independent adversarial review (round 35, + issue #1375): none of `{`, `}`, `while`, `until`, `for`, `select`, + `if`, `do`, `then`, `done`, `fi` were recognized at all -- a `{...}` + brace group or a `while`/`until`/`for`/`select`/`if` compound command + forks exactly like a subshell when the group AS A WHOLE is piped or + backgrounded, but isolation is conditional on that outer boundary, + never automatic the way a bare `(...)` subshell is -- see + `_segment_indices_isolated_by_a_piped_or_backgrounded_compound_ + group`'s own docstring for the full live-verified bypass, its + negative controls, and the position-0-only close-matching design + that keeps a literal argument like `echo fi` from desyncing + detection of a real, later group.""" raw = _raw_segments_with_boundaries(tokens) + group_isolated = _segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) result: list[tuple[list[str], bool]] = [] for i, (seg, depth, terminator) in enumerate(raw): if not seg: continue preceding = raw[i - 1][2] if i > 0 else None - isolated = depth > 0 or terminator in ("|", "&") or preceding == "|" or _seg_has_a_scope_localizing_keyword(seg) + isolated = ( + depth > 0 + or terminator in ("|", "&") + or preceding == "|" + or _seg_has_a_scope_localizing_keyword(seg) + or i in group_isolated + ) result.append((seg, isolated)) return result diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index 6be4f7dd..cf83ab14 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -473,6 +473,32 @@ def assert_allowed(command: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); coproc { echo hi; }; wait; VERB=safe; $TOOL $VERB foo", "real-top-level-static-clear-after-a-harmless-coproc-stays-allowed", ), + # No-over-correction guards for the thirty-fifth-round group-isolation + # fix (issue #1375): a bare `{...}`/`if`/`for` compound command with NO + # trailing `&`/`|` genuinely LEAKS its assignments to the parent shell + # in real bash (confirmed live: each variant's stand-in `uv` call + # captures `uv called with: safe foo`, not `install foo`) -- these must + # stay allowed exactly as before this round's fix. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { VERB=safe; }; $TOOL $VERB foo", + "bare-brace-group-with-no-trailing-background-or-pipe-stays-allowed", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); if true; then VERB=safe; fi; $TOOL $VERB foo", + "bare-if-with-no-trailing-background-or-pipe-stays-allowed", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); for i in 1; do VERB=safe; done; $TOOL $VERB foo", + "bare-for-loop-with-no-trailing-background-or-pipe-stays-allowed", + ), + ( + "TOOL=uv; VERB=safe; { echo hi; } & wait; $TOOL $VERB foo", + "unrelated-harmless-backgrounded-brace-group-alongside-a-never-poisoned-name-stays-allowed", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { echo hi; } & wait; VERB=safe; $TOOL $VERB foo", + "real-top-level-static-clear-after-a-harmless-backgrounded-brace-group-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1316,6 +1342,50 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: 'M=safe; M=$(echo POST); f() { $"local" M=GET; }; f; gh api repos/o/r/pulls/1/merge -X $M', "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-dollar-quoted-local", ), + # Found live by Step 8 independent review, thirty-fifth round (issue + # #1375): none of `{`, `}`, `while`, `do`, `done`, `until`, `for`, + # `select`, `if`, `then`, `fi` were recognized by the scope-isolation + # check at all -- but a `{...}` brace group or a `while`/`until`/ + # `for`/`select`/`if` compound command, backgrounded or piped AS A + # WHOLE, forks exactly like a subshell. Confirmed live via a stand-in + # `uv`/`gh` binary on PATH that each genuinely runs the dangerous + # command, NOT the group-scoped distractor value. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { VERB=safe; } & wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-backgrounded-brace-group", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { VERB=safe; } | cat; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-piped-brace-group", + ), + ( + "M=safe; M=$(echo POST); { true; M=GET; } & wait; gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-backgrounded-brace-group", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); while true; do VERB=safe; break; done | cat; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-piped-while-loop", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); if true; then VERB=safe; fi & wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-backgrounded-if", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); for i in 1; do VERB=safe; done & wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-backgrounded-for-loop", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { { VERB=safe; }; } | cat; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-doubly-nested-piped-brace-group", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); echo x | { true; VERB=safe; }; wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-pipe-receiving-groups-second-statement", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { echo fi; VERB=safe; } & wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-brace-group-containing-a-literal-fi-argument", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index a803038e..dd834b46 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2771,6 +2771,120 @@ def test_seg_has_a_scope_localizing_keyword_matches_model_for_a_dollar_quoted_ke assert checker._seg_has_a_scope_localizing_keyword([f"{name}={value}"]) is False +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_marks_a_backgrounded_brace_group() -> None: + """CRITICAL bypass regression pin (round-35 independent review, issue + #1375): a `{...}` brace group forks as one unit when backgrounded -- + a plain static assignment anywhere inside it can never reach the + parent shell's own copy of the name.""" + raw = checker._raw_segments_with_boundaries(["{", "VERB=safe", ";", "}", "&", "wait"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["{", "VERB=safe"] in isolated_segments + assert ["}"] in isolated_segments + assert ["wait"] not in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_marks_a_piped_brace_group() -> None: + raw = checker._raw_segments_with_boundaries(["{", "VERB=safe", ";", "}", "|", "cat"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["{", "VERB=safe"] in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_marks_a_piped_while_loop() -> None: + raw = checker._raw_segments_with_boundaries( + ["while", "true", ";", "do", "VERB=safe", ";", "break", ";", "done", "|", "cat"] + ) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["do", "VERB=safe"] in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_marks_a_backgrounded_if() -> None: + raw = checker._raw_segments_with_boundaries(["if", "true", ";", "then", "VERB=safe", ";", "fi", "&", "wait"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["then", "VERB=safe"] in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_marks_a_backgrounded_for_loop() -> None: + raw = checker._raw_segments_with_boundaries( + ["for", "i", "in", "1", ";", "do", "VERB=safe", ";", "done", "&", "wait"] + ) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["do", "VERB=safe"] in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_marks_a_doubly_nested_piped_group() -> None: + """The double-open `{ {` case: bash's own grammar places no + requirement that a nested group's own opener be separated from an + enclosing group's opener by anything but whitespace, so both land in + the SAME raw segment -- the isolation scan must still count both.""" + raw = checker._raw_segments_with_boundaries(["{", "{", "VERB=safe", ";", "}", ";", "}", "|", "cat"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["{", "{", "VERB=safe"] in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_marks_a_pipe_receiving_groups_later_statement() -> ( + None +): + """The group is the RECEIVING side of a pipe, and the assignment is + not even the group's own first statement -- must still be isolated.""" + raw = checker._raw_segments_with_boundaries(["echo", "x", "|", "{", "true", ";", "VERB=safe", ";", "}"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["VERB=safe"] in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_survives_a_literal_close_keyword_argument() -> ( + None +): + """SAFETY regression pin (round-35 own design review): an earlier + stack-based design that scanned every token position (not just + position 0) for a close keyword desynced against a literal, non- + syntactic argument like `fi` inside `echo fi`'s own segment -- it + popped the real group's own stack entry against that spurious match, + leaving the REAL closing `}` unmatched and the assignment wrongly + left unisolated. The position-0-only close check must not reopen + this: `echo fi` inside the group must never desync detection of the + group's own real, later `}`.""" + raw = checker._raw_segments_with_boundaries(["{", "echo", "fi", ";", "VERB=safe", ";", "}", "&", "wait"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["VERB=safe"] in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_allows_a_bare_brace_group() -> None: + """No over-correction: a bare `{...}`/`if`/`for` with NO trailing + `&`/`|` genuinely LEAKS its assignments to the parent shell in real + bash -- must not be marked isolated.""" + raw = checker._raw_segments_with_boundaries(["{", "VERB=safe", ";", "}", ";", "echo", "done"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + assert result == set() + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_VALUES) +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_matches_model_for_a_backgrounded_group( + name: str, value: str +) -> None: + """Model-based, exercising `_segment_indices_isolated_by_a_piped_or_ + backgrounded_compound_group` directly (issue #1178's own detection- + logic property-coverage requirement): for ANY identifier assigned a + value inside a `{...}` brace group that is itself backgrounded, the + segment carrying that assignment is included in the isolated set -- + and the same group with NO trailing `&`/`|` is not.""" + backgrounded = checker._raw_segments_with_boundaries(["{", f"{name}={value}", ";", "}", "&", "wait"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(backgrounded) + isolated_segments = [backgrounded[i][0] for i in sorted(result)] + assert ["{", f"{name}={value}"] in isolated_segments + + bare = checker._raw_segments_with_boundaries(["{", f"{name}={value}", ";", "}"]) + assert checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(bare) == set() + + def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_subshell_clear() -> None: """CRITICAL bypass regression pin (round-31 independent review, issue #1375): a static reassignment written INSIDE a `(...)` subshell @@ -2886,6 +3000,69 @@ def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_dollar_qu assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_backgrounded_brace_group_clear() -> None: + """CRITICAL bypass regression pin (round-35 independent review, issue + #1375): a `{...}` brace group forks as one unit when backgrounded -- + a static reassignment inside it never reaches the parent shell's own + copy of the name.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "{", "VERB=safe", ";", "}", "&", "wait"] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_piped_while_loop_clear() -> None: + """Same round, the `while`-loop-as-a-piped-unit counterpart.""" + tokens = [ + "VERB=harmless", + ";", + "VERB=$(echo install)", + ";", + "while", + "true", + ";", + "do", + "VERB=safe", + ";", + "break", + ";", + "done", + "|", + "cat", + ] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_allows_a_real_top_level_clear_after_a_harmless_brace_group() -> None: + """No over-correction: an EARLIER, harmless (bare, not backgrounded or + piped) brace group must not block a LATER, genuine top-level static + reassignment from clearing poisoning normally.""" + tokens = ["VERB=harmless", ";", "VERB=$(echo install)", ";", "{", "true", ";", "}", ";", "VERB=safe"] + assert checker._names_reassigned_from_a_static_value(tokens) == set() + + +@_PROPERTIES +@given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, isolated_value=_VALUES) +def test_names_reassigned_from_a_static_value_matches_model_for_a_backgrounded_brace_group_clear_attempt( + name: str, static_value: str, dynamic_value: str, isolated_value: str +) -> None: + """Model-based: for ANY identifier reassigned static -> dynamic and + then given a static value ONLY inside a backgrounded `{...}` brace + group, the name stays poisoned regardless of the group's own + assigned value.""" + tokens = [ + f"{name}={static_value}", + ";", + f"{name}=$({dynamic_value})", + ";", + "{", + f"{name}={isolated_value}", + ";", + "}", + "&", + "wait", + ] + assert name in checker._names_reassigned_from_a_static_value(tokens) + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, final_static_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model_for_a_static_dynamic_static_sequence( @@ -3102,6 +3279,28 @@ def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_dolla assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_backgrounded_brace_group() -> None: + """CRITICAL bypass regression pin (round-35 independent review, issue + #1375): a `{...}` brace group forks as one unit when backgrounded -- + a static reassignment inside it never reaches the parent shell's own + copy of the name -- must not clear an append-poisoned candidate.""" + tokens = ["VERB=inst", "VERB+=all", ";", "{", "VERB=safe", ";", "}", "&", "wait"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_piped_if() -> None: + tokens = ["VERB=inst", "VERB+=all", ";", "if", "true", ";", "then", "VERB=safe", ";", "fi", "|", "cat"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_clears_after_a_harmless_bare_brace_group() -> None: + """No over-correction: a bare (not backgrounded or piped) brace group + genuinely leaks its assignment to the parent -- a name reassigned + static this way must be cleared normally.""" + tokens = ["VERB=inst", "VERB+=all", ";", "{", "VERB=safe", ";", "}"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + def test_names_cleared_by_a_later_static_reassignment_clears_after_a_printf_v() -> None: tokens = ["printf", "-v", "VERB", "%s", "install", ";", "VERB=safe"] assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} @@ -3676,6 +3875,137 @@ def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_coproc() assert verdict.deny is False +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_backgrounded_brace_group_clear() -> ( + None +): + """CRITICAL bypass regression pin (round-35 independent review, + issue #1375): a `{...}` brace group forks as one unit when + backgrounded, exactly like `cmd &` -- with no non-isolating usage + from that boundary alone. Confirmed live via a stand-in `uv` binary + on PATH that this genuinely runs `uv install foo`, NOT `safe foo`.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); { VERB=safe; } & wait; $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_piped_brace_group_clear() -> None: + """Same round, the piped counterpart: every stage of a `|` pipeline + forks its own subshell, including a brace group used as a stage.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); { VERB=safe; } | cat; $TOOL $VERB foo") + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_backgrounded_brace_group_clear() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`. + Confirmed live via a stand-in `gh` binary on PATH that this + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST`, a genuine + unreviewed write.""" + verdict = checker.classify("M=safe; M=$(echo POST); { true; M=GET; } & wait; gh api repos/o/r/pulls/1/merge -X $M") + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_piped_while_loop_clear() -> None: + """Same round, the `while`-loop-as-a-piped-unit counterpart -- + absent `shopt -s lastpipe`, EVERY pipeline stage forks, including + the loop itself when it is one.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); while true; do VERB=safe; break; done | cat; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_backgrounded_if_clear() -> None: + """Same round, the `if` counterpart.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); if true; then VERB=safe; fi & wait; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_backgrounded_for_loop_clear() -> None: + """Same round, the `for` counterpart.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); for i in 1; do VERB=safe; done & wait; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_doubly_nested_piped_group_clear() -> ( + None +): + """Same round, the doubly-nested counterpart: bash's own grammar + places no requirement that a nested group's own opener be separated + from an enclosing group's opener by anything but whitespace.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { { VERB=safe; }; } | cat; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_pipe_receiving_groups_later_statement() -> ( + None +): + """Same round: the group is the RECEIVING side of a pipe, and the + assignment is not even the group's own first statement.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); echo x | { true; VERB=safe; }; wait; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_group_containing_a_literal_fi_argument() -> ( + None +): + """SAFETY regression pin (round-35 own design review): a literal, + non-syntactic `fi` argument (inside `echo fi`) sitting lexically + inside the same brace group must not desync detection of the + group's own real, later `}`.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { echo fi; VERB=safe; } & wait; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_allows_a_bare_brace_group_with_no_trailing_background_or_pipe() -> None: + """No over-correction: a bare `{...}` with NO trailing `&`/`|` + genuinely LEAKS its assignment to the parent shell in real bash + (confirmed live: the stand-in `uv` call captures `uv called with: + safe foo`, not `install foo`) -- must stay allowed.""" + verdict = checker.classify("TOOL=uv; VERB=harmless; VERB=$(echo install); { VERB=safe; }; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_bare_if_with_no_trailing_background_or_pipe() -> None: + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); if true; then VERB=safe; fi; $TOOL $VERB foo" + ) + assert verdict.deny is False + + +def test_classify_allows_a_bare_for_loop_with_no_trailing_background_or_pipe() -> None: + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); for i in 1; do VERB=safe; done; $TOOL $VERB foo" + ) + assert verdict.deny is False + + +def test_classify_allows_an_unrelated_harmless_backgrounded_brace_group_alongside_a_never_poisoned_name() -> None: + """No over-correction: an ordinary, unrelated backgrounded brace + group elsewhere in the command must not spuriously deny a command + whose watched name was never poisoned at all.""" + verdict = checker.classify("TOOL=uv; VERB=safe; { echo hi; } & wait; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_backgrounded_brace_group() -> None: + """No over-correction: a harmless, UNRELATED backgrounded brace + group earlier in the command must not block a LATER, genuine + top-level static reassignment from clearing poisoning normally.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); { echo hi; } & wait; VERB=safe; $TOOL $VERB foo" + ) + assert verdict.deny is False + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 3583a70e3ad4a16598b8918c7847bfbebcc5f053 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 03:18:19 +0000 Subject: [PATCH 41/46] fix(hooks): recognize case/esac as scope-isolating and fix its depth corruption A fresh, independent adversarial review of this PR's current head (round 36) found case/esac was entirely absent from the scope-isolation machinery rounds 31-35 built for its sibling compound commands, and found a second, distinct bug in how its own syntax interacts with the pre-existing (...)-subshell depth tracking. Finding 1: case...esac is bash's remaining compound-command form -- it forks as one unit when the whole statement is piped or backgrounded, exactly like the brace-group/while/until/for/select/if forms round 35 already closed -- but neither "case" nor "esac" were ever added to the group-isolation keyword sets. Independently reproduced live via classify() and real bash execution with stand-in uv/gh binaries on PATH: a backgrounded case around a static reassignment resolved to deny=False even though real bash genuinely runs the earlier, still- dynamic value's own dangerous command; the piped form and the _rule_gh_api_write counterpart both reproduce identically. A bare case...esac with no trailing background or pipe boundary genuinely leaks to the parent in real bash and stays correctly allowed post-fix. Closed by adding "case"/"esac" to the existing keyword sets -- both already satisfy the position-0/leading-open-run detection every other keyword pair relies on, so no change to the bracket-matching logic itself was needed. Finding 2: a case statement's own pattern arm ends with a bare ")" (e.g. "1) ...", "a|b) ...") that is lexically indistinguishable, at the token level, from a subshell-closing ")" -- but unlike every other paired delimiter this classifier tracks, a case pattern's closing paren has no corresponding open token at all. The pre-fix code decremented the tracked (...)-nesting depth unconditionally on every bare ")", so a case nested inside a genuinely enclosing subshell prematurely "closed" that subshell's own tracked depth on the case's first pattern arm, long before the real closing paren was reached -- reproducing with no background or pipe operator involved at all, purely through ordinary subshell nesting. Confirmed live that the outer subshell genuinely still isolates the assignment in real bash. Closed by tracking, per currently-open case block (a stack, since case statements nest), whether the next bare ")" is that block's own pattern terminator rather than a real subshell close: armed once, right after that case's own first "in" keyword; disarmed the instant a ")" is consumed as a pattern terminator (still creating a segment boundary, but leaving depth unchanged); re-armed on the next double-semicolon while the block stays open; and popped off the stack at "esac". A companion safety concern was checked during this round's own design review: a per-block flag ensures only the case's own first "in" arms the pattern-close expectation, so a LATER, unrelated "in" lexically inside an arm's body (from a nested for-loop or select-loop) is never mistaken for the case's own -- confirmed live that a real subshell genuinely enclosing such a nested loop stays correctly isolated, not desynced by the inner loop's own "in" keyword. Regression tests added at every established layer: unit tests for both the depth-tracking fix and the group-isolation fix directly (including the nested-real-subshell-inside-an-arm and nested-for-in-desync safety cases), a Hypothesis property test exercising the group-isolation function by name (issue #1178's own coverage requirement), unit and classify()-level tests for both consumer functions' new isolation classes plus false-positive controls (a bare case, an unrelated backgrounded case, a real top-level clear after a harmless one, a genuine subshell nested inside a case arm that must still leak normally after its own pattern close), and wrapper-level end-to-end pins against real stand-in uv/gh binaries. Full gate suite green: ruff check/format, mypy, the detection-logic property-coverage gate (run against the uncommitted working tree, no HEAD ref), and no coverage regression on the classifier file (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-36 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 117 ++++++- hooks/test_gitapex_check_bash_safety.py | 64 ++++ ...st_gitapex_check_bash_safety_properties.py | 289 ++++++++++++++++++ 3 files changed, 464 insertions(+), 6 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 4271554f..9add1fe7 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4129,12 +4129,76 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in Closed by recognizing every token in `_SUBSHELL_OPEN_TOKENS` (not just a bare `(`) as a depth-opening boundary -- `<(`/`>(` each still pair with an ordinary bare `)` close exactly like a bare `(` does, - so no change to the closing side was needed.""" + so no change to the closing side was needed. + + CRITICAL bypass found by independent adversarial review (round 36, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: every bash `case` statement's own pattern arm ends + with a bare `)` (`1) ...`, `a|b) ...`) that is lexically + INDISTINGUISHABLE, at the token level, from a subshell-closing `)` + -- there is no corresponding OPEN token for it at all, unlike + `<(`/`>(` above. The pre-round-36 code decremented `depth` + unconditionally on every bare `)`, so a `case` nested inside a + GENUINELY enclosing `(...)` subshell prematurely "closed" that + subshell's own tracked depth on the case's first pattern arm, long + before the real closing `)` was reached. `TOOL=uv; VERB=harmless; + VERB=$(echo install); ( case 1 in 1) true ;; esac; VERB=safe ); + $TOOL $VERB foo` resolved to `deny=False` even though real bash + genuinely runs `uv install foo` (confirmed live via a stand-in `uv` + binary on PATH: captured argv `uv called with: install foo`) -- the + outer `(...)` genuinely still isolates `VERB=safe` in real bash + (confirmed live: `VERB=install; ( case 1 in 1) true ;; esac; + VERB=safe ); echo "VERB is now: $VERB"` prints `VERB is now: + install`), but the case arm's own phantom `)` had already + decremented the tracked depth to 0 by the time `VERB=safe` was + reached, wrongly treating it as a top-level assignment. No `&`/`|` + is needed to reproduce this -- ordinary subshell nesting alone + triggers it, distinct from `_segment_indices_isolated_by_a_piped_ + or_backgrounded_compound_group`'s own round-35/36 pipe/background + mechanism (see that function's own docstring for the SEPARATE, + companion round-36 fix closing `case`/`esac` there). + + Closed by tracking, per currently-open `case` block (a stack, since + `case` statements nest), whether the NEXT bare `)` is that case + block's own pattern-arm terminator rather than a real subshell + close: armed once, right after that case's own first `in` keyword + (a per-block `case_seen_in` flag ensures a LATER, unrelated `in` + lexically inside an arm's body -- e.g. from a nested `for i in + ...`/`select i in ...` -- is never mistaken for the case's own, + since only the FIRST `in` after each `case` push counts); disarmed + the instant a `)` is consumed as that arm's own pattern terminator + (segmenting exactly as before, but leaving `depth` UNCHANGED rather + than decrementing it); re-armed on the next `;;` (detected as two + consecutive bare `;` tokens, confirmed via `tokenize()`: `1) VERB= + safe ;; esac` produces `[..., "VERB=safe", ";", ";", "esac"]`, never + a single fused token) while that case block is still open; and + popped off the stack entirely at `esac`, regardless of its own + armed/disarmed state at that point. A `(...)` subshell genuinely + nested INSIDE a case arm's own body (after its pattern-terminating + `)` has already been consumed, so the block's own flag is + disarmed) is unaffected -- its own bare `)` decrements depth + normally, exactly as before this fix.""" segments: list[list[str]] = [[]] seg_depths: list[int] = [0] terminators: list[str | None] = [] depth = 0 + case_awaiting_pattern_close: list[bool] = [] + case_seen_in: list[bool] = [] + prev_tok: str | None = None for tok in tokens: + if tok == "case": + case_awaiting_pattern_close.append(False) + case_seen_in.append(False) + elif tok == "in" and case_seen_in and not case_seen_in[-1]: + case_seen_in[-1] = True + case_awaiting_pattern_close[-1] = True + elif tok == "esac" and case_awaiting_pattern_close: + case_awaiting_pattern_close.pop() + case_seen_in.pop() + elif tok == ";" and case_awaiting_pattern_close and prev_tok == ";": + case_awaiting_pattern_close[-1] = True + if tok in _SUBSHELL_OPEN_TOKENS: terminators.append(tok) depth += 1 @@ -4142,7 +4206,10 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in seg_depths.append(depth) elif tok == ")": terminators.append(tok) - depth = max(depth - 1, 0) + if case_awaiting_pattern_close and case_awaiting_pattern_close[-1]: + case_awaiting_pattern_close[-1] = False + else: + depth = max(depth - 1, 0) segments.append([]) seg_depths.append(depth) elif tok in _SINGLE_OPS or tok in _MULTI_OPS: @@ -4151,6 +4218,7 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in seg_depths.append(depth) else: segments[-1].append(tok) + prev_tok = tok terminators.append(None) return list(zip(segments, seg_depths, terminators, strict=True)) @@ -4182,8 +4250,8 @@ def _seg_has_a_scope_localizing_keyword(seg: list[str]) -> bool: return False -_GROUP_OPEN_KEYWORDS = frozenset({"{", "while", "until", "for", "select", "if"}) -_GROUP_CLOSE_KEYWORDS = frozenset({"}", "done", "fi"}) +_GROUP_OPEN_KEYWORDS = frozenset({"{", "while", "until", "for", "select", "if", "case"}) +_GROUP_CLOSE_KEYWORDS = frozenset({"}", "done", "fi", "esac"}) """The compound-command delimiters `_segment_indices_isolated_by_a_ piped_or_backgrounded_compound_group` bracket-matches. Unlike `_SUBSHELL_ OPEN_TOKENS`, none of these tokens themselves make their own content @@ -4193,7 +4261,13 @@ def _seg_has_a_scope_localizing_keyword(seg: list[str]) -> bool: here depends entirely on whether the GROUP AS A WHOLE is itself a pipeline stage or a backgrounded job, the same test already applied to an ordinary segment, evaluated once at the group's own matching open/close -pair. Added by round 35 (issue #1375).""" +pair. Added by round 35 (`{`/`while`/`until`/`for`/`select`/`if`) and +round 36 (`case`/`esac`), issue #1375 -- see this function's own +docstring for the round-36 `case`/`esac` bypass and why `case` still +satisfies the leading-contiguous-open-run detection this function relies +on (a `case WORD in` segment's own leading token is always `case`, with +the case's own pattern text and `in` keyword following in the SAME +segment, never opening a nested level of their own).""" def _segment_indices_isolated_by_a_piped_or_backgrounded_compound_group( @@ -4289,7 +4363,38 @@ def _segment_indices_isolated_by_a_piped_or_backgrounded_compound_group( non-final-or-final pipe stage, or because it is backgrounded), OR when the raw segment immediately preceding the group's own opening raw segment terminates with `|` (the group is itself the RECEIVING - side of a pipe -- case H above).""" + side of a pipe -- case H above). + + ONE further gap found by independent adversarial review (round 36, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: `case ... esac` is bash's remaining compound-command + form -- it forks exactly like `{...}`/`while`/`until`/`for`/ + `select`/`if` when the ENTIRE statement is piped or backgrounded -- + but neither `case` nor `esac` were ever added to `_GROUP_OPEN_ + KEYWORDS`/`_GROUP_CLOSE_KEYWORDS`. `TOOL=uv; VERB=harmless; VERB= + $(echo install); case 1 in 1) VERB=safe ;; esac & wait; $TOOL $VERB + foo` resolved to `deny=False` even though real bash genuinely runs + `uv install foo` (confirmed live via a stand-in `uv` binary on PATH: + captured argv `uv called with: install foo`); the piped form (`esac + | cat`) and the `_rule_gh_api_write` counterpart both reproduce + identically. A bare `case ... esac` with NO trailing `&`/`|` + genuinely leaks to the parent in real bash (confirmed live: `uv + called with: safe foo`) and stays correctly allowed post-fix, the + same negative-control shape already established above for the + other five keywords. Closed by adding `case`/`esac` to the two + keyword sets -- `case` satisfies the leading-contiguous-open-run + detection the same way every other opener does (a `case WORD in` + segment's own leading token is always `case`, with the WORD and + `in` keyword following in the SAME segment, never opening a nested + level of their own), and `esac` satisfies the position-0-of-its-own- + segment requirement the same way `}`/`done`/`fi` do (always preceded + by a `;`/`;;` boundary in valid bash). A SEPARATE, companion + round-36 bug in the `(...)`-subshell depth tracking this function's + own `raw` parameter is built from -- a case arm's own pattern- + terminating `)` being mistaken for a real subshell close -- is fixed + in `_raw_segments_with_boundaries` itself; see that function's own + docstring.""" stack: list[int] = [] isolated: set[int] = set() for i, (seg, _depth, terminator) in enumerate(raw): diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index cf83ab14..db7ab473 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -499,6 +499,32 @@ def assert_allowed(command: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); { echo hi; } & wait; VERB=safe; $TOOL $VERB foo", "real-top-level-static-clear-after-a-harmless-backgrounded-brace-group-stays-allowed", ), + # No-over-correction guards for the thirty-sixth-round case/esac + # group-isolation fix (issue #1375): a bare `case ... esac` with NO + # trailing `&`/`|` genuinely LEAKS its assignment to the parent shell + # in real bash (confirmed live: `uv called with: safe foo`), a + # GENUINE `(...)` subshell nested inside a case arm's own body must + # still decrement depth normally once that arm's own pattern- + # terminating `)` has already been consumed, and an unrelated, + # never-poisoned name must not be spuriously denied by a harmless + # case elsewhere in the command. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) VERB=safe ;; esac; $TOOL $VERB foo", + "bare-case-with-no-trailing-background-or-pipe-stays-allowed", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); " + "case 1 in 1) ( VERB=x ); true ;; esac; VERB=safe; $TOOL $VERB foo", + "real-subshell-nested-inside-a-case-arm-that-genuinely-leaks-after-stays-allowed", + ), + ( + "TOOL=uv; VERB=safe; case 1 in 1) echo hi ;; esac & wait; $TOOL $VERB foo", + "unrelated-harmless-backgrounded-case-alongside-a-never-poisoned-name-stays-allowed", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) echo hi ;; esac & wait; VERB=safe; $TOOL $VERB foo", + "real-top-level-static-clear-after-a-harmless-backgrounded-case-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1386,6 +1412,44 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); { echo fi; VERB=safe; } & wait; $TOOL $VERB foo", "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-brace-group-containing-a-literal-fi-argument", ), + # Found live by Step 8 independent review, thirty-sixth round (issue + # #1375): `case ... esac` is bash's remaining compound-command form + # -- it forks as one unit when backgrounded or piped, exactly like + # `{...}`/`while`/`until`/`for`/`select`/`if` (round 35), but + # `case`/`esac` were never added to the group-isolation keyword + # sets. Separately, a case arm's own pattern-terminating `)` is + # lexically indistinguishable from a subshell-closing `)` and was + # unconditionally decrementing `(...)`-nesting depth, corrupting + # tracking for a genuinely enclosing real subshell -- reproducing + # with no `&`/`|` at all. Confirmed live via a stand-in `uv`/`gh` + # binary on PATH that each genuinely runs the dangerous command, + # NOT the case-scoped distractor value. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) VERB=safe ;; esac & wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-backgrounded-case", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) VERB=safe ;; esac | cat; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-piped-case", + ), + ( + "M=safe; M=$(echo POST); case 1 in 1) M=GET ;; esac & wait; gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-backgrounded-case", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); ( case 1 in 1) true ;; esac; VERB=safe ); $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-case-in-subshell-depth-corruption", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); " + "( case 1 in 1) for i in 1; do true; done ;; esac; VERB=safe ); $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-case-in-subshell-with-a-nested-for-in-not-desyncing", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); " + "case A in x) case B in y) VERB=safe;; esac ;; esac | cat; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-nested-case-whose-outer-is-piped", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index dd834b46..87837461 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2706,6 +2706,109 @@ def test_raw_segments_with_boundaries_process_substitution_does_not_corrupt_an_e assert result[5] == ([], 0, None) +def test_raw_segments_with_boundaries_treats_a_case_pattern_close_as_not_a_subshell_close() -> None: + """CRITICAL bypass regression pin (round-36 independent review, issue + #1375): a `case` arm's own pattern-terminating `)` is lexically + indistinguishable from a subshell-closing `)`, but must NOT + decrement `(...)`-nesting depth -- a case nested inside a GENUINELY + enclosing subshell must keep that subshell's own depth tracking + intact through the case's own pattern arms.""" + tokens = ["(", "case", "1", "in", "1", ")", "true", ";", ";", "esac", ";", "VERB=safe", ")"] + result = checker._raw_segments_with_boundaries(tokens) + assert result[0] == ([], 0, "(") + assert result[1] == (["case", "1", "in", "1"], 1, ")") + assert result[5] == (["VERB=safe"], 1, ")") + assert result[6] == ([], 0, None) + + +def test_raw_segments_with_boundaries_a_real_subshell_nested_inside_a_case_arm_still_decrements() -> None: + """No over-correction: a GENUINE `(...)` subshell lexically inside a + case arm's own body (after that arm's pattern-terminating `)` has + already been consumed) must still decrement depth normally.""" + tokens = ["case", "1", "in", "1", ")", "(", "VERB=x", ")", ";", "true", ";", ";", "esac"] + result = checker._raw_segments_with_boundaries(tokens) + depths_by_segment = [depth for seg, depth, _term in result if seg] + assert depths_by_segment == [0, 1, 0, 0] + + +def test_raw_segments_with_boundaries_a_nested_for_in_inside_a_case_arm_does_not_desync_pattern_tracking() -> None: + """SAFETY regression pin (round-36 own design review): only the + case's own FIRST `in` (right after `case WORD`) may arm the + pattern-close expectation -- a LATER, unrelated `in` from a nested + `for i in ...`/`select i in ...` lexically inside an arm's body must + never be mistaken for the case's own, or a genuinely enclosing + subshell's own real closing `)` would be wrongly swallowed as a + phantom case-pattern close instead of decrementing depth.""" + tokens = [ + "(", + "case", + "1", + "in", + "1", + ")", + "for", + "i", + "in", + "1", + ";", + "do", + "true", + ";", + "done", + ";", + ";", + "esac", + ")", + ] + result = checker._raw_segments_with_boundaries(tokens) + assert result[-1] == ([], 0, None) + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_marks_a_backgrounded_case() -> None: + """CRITICAL bypass regression pin (round-36 independent review, issue + #1375): `case ... esac` is bash's remaining compound-command form -- + it forks as one unit when backgrounded, exactly like `{...}`/ + `while`/`until`/`for`/`select`/`if` (round 35), but `case`/`esac` + were never added to `_GROUP_OPEN_KEYWORDS`/`_GROUP_CLOSE_KEYWORDS`.""" + raw = checker._raw_segments_with_boundaries( + ["case", "1", "in", "1", ")", "VERB=safe", ";", ";", "esac", "&", "wait"] + ) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + isolated_segments = [raw[i][0] for i in sorted(result)] + assert ["VERB=safe"] in isolated_segments + + +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_allows_a_bare_case() -> None: + """No over-correction: a bare `case ... esac` with NO trailing + `&`/`|` genuinely LEAKS its assignments to the parent shell in real + bash -- must not be marked isolated.""" + raw = checker._raw_segments_with_boundaries(["case", "1", "in", "1", ")", "VERB=safe", ";", ";", "esac"]) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(raw) + assert result == set() + + +@_PROPERTIES +@given(name=_IDENTIFIERS, value=_VALUES) +def test_segment_indices_isolated_by_a_piped_or_backgrounded_compound_group_matches_model_for_a_backgrounded_case( + name: str, value: str +) -> None: + """Model-based, exercising `_segment_indices_isolated_by_a_piped_or_ + backgrounded_compound_group` directly (issue #1178's own detection- + logic property-coverage requirement): for ANY identifier assigned a + value inside a `case ... esac` that is itself backgrounded, the + segment carrying that assignment is included in the isolated set -- + and the same case with NO trailing `&`/`|` is not.""" + backgrounded = checker._raw_segments_with_boundaries( + ["case", "1", "in", "1", ")", f"{name}={value}", ";", ";", "esac", "&", "wait"] + ) + result = checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(backgrounded) + isolated_segments = [backgrounded[i][0] for i in sorted(result)] + assert [f"{name}={value}"] in isolated_segments + + bare = checker._raw_segments_with_boundaries(["case", "1", "in", "1", ")", f"{name}={value}", ";", ";", "esac"]) + assert checker._segment_indices_isolated_by_a_piped_or_backgrounded_compound_group(bare) == set() + + def test_segment_tokens_with_scope_isolation_marks_a_declare_declaration_isolated() -> None: tokens = ["{", "declare", "VERB=safe"] result = checker._segment_tokens_with_scope_isolation(tokens) @@ -3039,6 +3142,60 @@ def test_names_reassigned_from_a_static_value_allows_a_real_top_level_clear_afte assert checker._names_reassigned_from_a_static_value(tokens) == set() +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_backgrounded_case_clear() -> None: + """CRITICAL bypass regression pin (round-36 independent review, issue + #1375): `case ... esac` forks as one unit when backgrounded, exactly + like `{...}`/`while`/`until`/`for`/`select`/`if` -- a static + reassignment inside it never reaches the parent shell's own copy of + the name.""" + tokens = [ + "VERB=harmless", + ";", + "VERB=$(echo install)", + ";", + "case", + "1", + "in", + "1", + ")", + "VERB=safe", + ";", + ";", + "esac", + "&", + "wait", + ] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_case_in_subshell_depth_corruption() -> None: + """CRITICAL bypass regression pin (round-36 independent review, issue + #1375): a case arm's own pattern-terminating `)` must not be + mistaken for a real subshell close -- a static reassignment inside a + GENUINELY enclosing subshell around a `case` must still stay + poisoned even with no `&`/`|` involved at all.""" + tokens = [ + "VERB=harmless", + ";", + "VERB=$(echo install)", + ";", + "(", + "case", + "1", + "in", + "1", + ")", + "true", + ";", + ";", + "esac", + ";", + "VERB=safe", + ")", + ] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, isolated_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model_for_a_backgrounded_brace_group_clear_attempt( @@ -3301,6 +3458,23 @@ def test_names_cleared_by_a_later_static_reassignment_clears_after_a_harmless_ba assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} +def test_names_cleared_by_a_later_static_reassignment_does_not_clear_via_a_backgrounded_case() -> None: + """CRITICAL bypass regression pin (round-36 independent review, issue + #1375): `case ... esac` forks as one unit when backgrounded -- a + static reassignment inside it never reaches the parent shell's own + copy of the name -- must not clear an append-poisoned candidate.""" + tokens = ["VERB=inst", "VERB+=all", ";", "case", "1", "in", "1", ")", "VERB=safe", ";", ";", "esac", "&", "wait"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == set() + + +def test_names_cleared_by_a_later_static_reassignment_clears_after_a_harmless_bare_case() -> None: + """No over-correction: a bare (not backgrounded or piped) `case ... + esac` genuinely leaks its assignment to the parent -- a name + reassigned static this way must be cleared normally.""" + tokens = ["VERB=inst", "VERB+=all", ";", "case", "1", "in", "1", ")", "VERB=safe", ";", ";", "esac"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + def test_names_cleared_by_a_later_static_reassignment_clears_after_a_printf_v() -> None: tokens = ["printf", "-v", "VERB", "%s", "install", ";", "VERB=safe"] assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} @@ -4006,6 +4180,121 @@ def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_backgrou assert verdict.deny is False +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_backgrounded_case_clear() -> None: + """CRITICAL bypass regression pin (round-36 independent review, + issue #1375): `case ... esac` is bash's remaining compound-command + form -- it forks as one unit when backgrounded, exactly like + `{...}`/`while`/`until`/`for`/`select`/`if` (round 35), but + `case`/`esac` were never added to the group-isolation keyword sets. + Confirmed live via a stand-in `uv` binary on PATH that this + genuinely runs `uv install foo`, NOT `safe foo`.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) VERB=safe ;; esac & wait; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_piped_case_clear() -> None: + """Same round, the piped counterpart: every stage of a `|` pipeline + forks its own subshell, including a `case` used as a stage.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) VERB=safe ;; esac | cat; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_backgrounded_case_clear() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`. + Confirmed live via a stand-in `gh` binary on PATH that this + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST`, a genuine + unreviewed write.""" + verdict = checker.classify( + "M=safe; M=$(echo POST); case 1 in 1) M=GET ;; esac & wait; gh api repos/o/r/pulls/1/merge -X $M" + ) + assert verdict.deny is True + + +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_case_in_subshell_depth_corruption() -> ( + None +): + """CRITICAL bypass regression pin (round-36 independent review, + issue #1375): a `case` arm's own pattern-terminating `)` must not + be mistaken for a real subshell close -- reproduces with NO `&`/`|` + at all, purely by nesting a `case` inside a genuine `(...)` + subshell. Confirmed live via a stand-in `uv` binary on PATH that + the outer subshell genuinely still isolates `VERB=safe` in real + bash.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); ( case 1 in 1) true ;; esac; VERB=safe ); $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_allows_a_bare_case_with_no_trailing_background_or_pipe() -> None: + """No over-correction: a bare `case ... esac` with NO trailing + `&`/`|` genuinely LEAKS its assignment to the parent shell in real + bash (confirmed live: the stand-in `uv` call captures `uv called + with: safe foo`, not `install foo`) -- must stay allowed.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) VERB=safe ;; esac; $TOOL $VERB foo" + ) + assert verdict.deny is False + + +def test_classify_allows_a_real_subshell_nested_inside_a_case_arm_that_genuinely_leaks_after() -> None: + """No over-correction: a GENUINE `(...)` subshell lexically inside a + case arm's own body must still decrement depth normally once its + own pattern-terminating `)` has already been consumed -- confirmed + live that the later, real top-level `VERB=safe` genuinely leaks.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); " + "case 1 in 1) ( VERB=x ); true ;; esac; VERB=safe; $TOOL $VERB foo" + ) + assert verdict.deny is False + + +def test_classify_denies_despite_a_nested_for_in_inside_a_case_arm_not_desyncing_pattern_tracking() -> None: + """SAFETY regression pin (round-36 own design review): only the + case's own FIRST `in` may arm the pattern-close expectation -- a + LATER, unrelated `in` from a nested `for i in ...` lexically inside + an arm's body must never desync tracking of a genuinely enclosing + subshell's own real closing `)`.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); " + "( case 1 in 1) for i in 1; do true; done ;; esac; VERB=safe ); $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_nested_case_whose_outer_is_piped() -> None: + """Same round, the doubly-nested counterpart: an inner `case` that + is itself plain must not prevent the OUTER case's own piping from + isolating everything inside it.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); " + "case A in x) case B in y) VERB=safe;; esac ;; esac | cat; $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_allows_an_unrelated_harmless_backgrounded_case_alongside_a_never_poisoned_name() -> None: + """No over-correction: an ordinary, unrelated backgrounded `case` + elsewhere in the command must not spuriously deny a command whose + watched name was never poisoned at all.""" + verdict = checker.classify("TOOL=uv; VERB=safe; case 1 in 1) echo hi ;; esac & wait; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_backgrounded_case() -> None: + """No over-correction: a harmless, UNRELATED backgrounded `case` + earlier in the command must not block a LATER, genuine top-level + static reassignment from clearing poisoning normally.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) echo hi ;; esac & wait; VERB=safe; $TOOL $VERB foo" + ) + assert verdict.deny is False + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From 429a2c0ac24d27980bd4a48498688ccc37646137 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 03:34:30 +0000 Subject: [PATCH 42/46] fix(hooks): stop a case pattern's optional leading paren from inflating depth A fresh, independent adversarial review of this PR's current head (round 37) found a false-positive bug introduced by the round-36 fix for case/esac scope isolation. Bash's case syntax allows an OPTIONAL leading paren decorator on a pattern arm -- "(1) cmd ;;", "(1|2) cmd ;;" -- common, POSIX/ksh- compatible style, no shopt needed. That opening paren is lexically identical to a real subshell opener, so it hit the ordinary subshell- open handling and unconditionally incremented the tracked nesting depth. The round-36 fix only consulted its own case-pattern-tracking state on the CLOSING paren, never the opening one, so the same closing paren that correctly recognized itself as the arm's own pattern terminator (and correctly skipped decrementing) left the decorator's own phantom increment permanently unbalanced. Tracked depth stayed inflated by one for the rest of the token stream, which downstream means every segment after such a case block was wrongly treated as scope-isolated -- so a perfectly ordinary, genuinely top-level static reassignment after the case statement could never clear an earlier poisoning. Independently reproduced live via classify() and real bash execution with stand-in uv/gh binaries on PATH: a decorated case arm followed by a genuine top-level clearing reassignment resolved to deny=True even though real bash genuinely runs the harmless, cleared command; the alternation-pattern form and the gh api write-detector counterpart both reproduce identically. Traced that this bug can only ever inflate tracked depth relative to real bash's own true nesting, never deflate it, so it is confirmed a false positive (over-denial), not a bypass -- unlike every round-30-36 finding in this same area. Closed by tracking, per currently-open case block, whether its current pattern arm has consumed any real token yet since being armed. A bare opening paren is now treated as the harmless decorator, with its depth increment suppressed, only when it is the very first token of the current arm; every token consumed while a block is armed marks the arm as started immediately afterward, except the arm/re-arm setup tokens themselves. This deliberately leaves a rare, opt-in bash extglob pattern's own internal parens unaffected -- a narrower, disclosed residual (an over-denial on a feature requiring an explicit shopt, never a bypass) rather than a fully general case-pattern parser. Regression tests added at every established layer: unit tests for the depth-tracking fix directly (including a check that the fix does not disable the existing group-isolation mechanism for a genuinely backgrounded decorated case), unit and classify()-level tests for both consumer functions' new behavior plus the alternation-pattern and gh-api-write forms, and wrapper-level end-to-end pins against real stand-in uv/gh binaries (both the newly-allowed and the still-denied shapes). Full gate suite green: ruff check/format, mypy, the detection-logic property-coverage gate (already satisfied by existing coverage on the touched function), and no coverage regression on the classifier file (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-37 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 81 +++++++++++++- hooks/test_gitapex_check_bash_safety.py | 24 ++++ ...st_gitapex_check_bash_safety_properties.py | 103 ++++++++++++++++++ 3 files changed, 206 insertions(+), 2 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index 9add1fe7..def3adbb 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4178,32 +4178,106 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in nested INSIDE a case arm's own body (after its pattern-terminating `)` has already been consumed, so the block's own flag is disarmed) is unaffected -- its own bare `)` decrements depth - normally, exactly as before this fix.""" + normally, exactly as before this fix. + + FALSE-POSITIVE bug found by independent adversarial review (round + 37, issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH, in the round-36 fix above: bash's `case` syntax + allows an OPTIONAL leading `(` decorator on a pattern arm (`(1) + cmd ;;`, `(1|2) cmd ;;` -- common, POSIX/ksh-compatible style, no + `shopt` needed). That `(` is lexically identical to a real subshell + opener, so it hit the ordinary `_SUBSHELL_OPEN_TOKENS` branch and + unconditionally incremented `depth` -- but the round-36 state + machine only consulted `case_awaiting_pattern_close` on the + CLOSING `)`, never the opening `(`, so the SAME `)` that correctly + skipped decrementing (recognizing it as the arm's own pattern + terminator) left the phantom `+1` from the decorator's own `(` + permanently unbalanced -- `depth` stayed inflated by 1 for the rest + of the token stream (the `max(depth - 1, 0)` clamp only guards + against going negative, not against this kind of unmatched-open + drift). `TOOL=uv; VERB=inst; VERB+=all; case 1 in (1) true ;; esac; + VERB=safe; $TOOL $VERB foo` resolved to `deny=True` even though + real bash genuinely runs `uv safe foo` (confirmed live via a + stand-in `uv` binary on PATH: captured argv `uv called with: safe + foo`) -- the case block has nothing to do with `VERB`'s own later, + genuinely top-level clearing reassignment, which should have been + trusted normally; `_rule_gh_api_write` and the `(1|2)`-alternation + form both reproduce identically. Unlike every round-30-36 finding + in this same area, this is a FALSE POSITIVE (over-denial), not a + bypass -- confirmed by tracing that `depth` can only ever be + inflated relative to real bash's own true nesting by this bug + (never deflated, since a phantom `+1` from the decorator is simply + never balanced, and the clamp only prevents going negative), so + `isolated` can wrongly read `True` when it should be `False`, but + never the reverse. + + Closed by tracking, per currently-open case block, whether its + CURRENT pattern arm has consumed any real token yet since being + armed (`case_pattern_started`, reset to `False` on the same events + that arm/re-arm `case_awaiting_pattern_close`: the block's own + first `in`, and each subsequent `;;`) -- a bare `(` is treated as + the harmless decorator, and its would-be depth increment + suppressed, ONLY when it is the very first token of the current + arm (`case_awaiting_pattern_close[-1] and not case_pattern_started[ + -1]`); every token processed while a block is armed sets `case_ + pattern_started[-1] = True` immediately afterward (except the + arm/re-arm setup tokens themselves -- `case`/`in`/`esac`/the second + `;` of a `;;` -- which must not count as the pattern's own first + token or the decorator would never be recognized at all). This + correctly leaves an extglob pattern's own internal `(` (e.g. bash's + `@(foo|bar)` syntax, which requires `shopt -s extglob` and is off + by default) NOT specially suppressed, since its own leading `@` + token consumes the "first token" slot before its `(` is ever seen + -- a deliberately narrower, disclosed residual (an over-denial on a + rare, opt-in bash feature, never a bypass) rather than a fully + general case-pattern-syntax parser.""" segments: list[list[str]] = [[]] seg_depths: list[int] = [0] terminators: list[str | None] = [] depth = 0 case_awaiting_pattern_close: list[bool] = [] case_seen_in: list[bool] = [] + case_pattern_started: list[bool] = [] prev_tok: str | None = None for tok in tokens: + is_case_setup_token = False if tok == "case": case_awaiting_pattern_close.append(False) case_seen_in.append(False) + case_pattern_started.append(False) + is_case_setup_token = True elif tok == "in" and case_seen_in and not case_seen_in[-1]: case_seen_in[-1] = True case_awaiting_pattern_close[-1] = True + case_pattern_started[-1] = False + is_case_setup_token = True elif tok == "esac" and case_awaiting_pattern_close: case_awaiting_pattern_close.pop() case_seen_in.pop() + case_pattern_started.pop() + is_case_setup_token = True elif tok == ";" and case_awaiting_pattern_close and prev_tok == ";": case_awaiting_pattern_close[-1] = True + case_pattern_started[-1] = False + is_case_setup_token = True + + is_case_pattern_decorator_paren = ( + tok == "(" + and case_awaiting_pattern_close + and case_awaiting_pattern_close[-1] + and not case_pattern_started[-1] + ) - if tok in _SUBSHELL_OPEN_TOKENS: + if tok in _SUBSHELL_OPEN_TOKENS and not is_case_pattern_decorator_paren: terminators.append(tok) depth += 1 segments.append([]) seg_depths.append(depth) + elif tok in _SUBSHELL_OPEN_TOKENS: + terminators.append(tok) + segments.append([]) + seg_depths.append(depth) elif tok == ")": terminators.append(tok) if case_awaiting_pattern_close and case_awaiting_pattern_close[-1]: @@ -4218,6 +4292,9 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in seg_depths.append(depth) else: segments[-1].append(tok) + + if case_awaiting_pattern_close and case_awaiting_pattern_close[-1] and not is_case_setup_token: + case_pattern_started[-1] = True prev_tok = tok terminators.append(None) return list(zip(segments, seg_depths, terminators, strict=True)) diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index db7ab473..bbbb5cb2 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -525,6 +525,26 @@ def assert_allowed(command: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in 1) echo hi ;; esac & wait; VERB=safe; $TOOL $VERB foo", "real-top-level-static-clear-after-a-harmless-backgrounded-case-stays-allowed", ), + # FALSE-POSITIVE fix found live by Step 8 independent review, + # thirty-seventh round (issue #1375): bash's `case` syntax allows an + # OPTIONAL leading `(` decorator on a pattern arm (`(1) cmd ;;`, + # `(1|2) cmd ;;` -- common, POSIX/ksh-compatible style, no `shopt` + # needed) -- lexically identical to a real subshell opener, but the + # round-36 fix only consulted its own case-tracking state on the + # CLOSING paren, never the opening one, so the decorator's own + # phantom depth increment was never balanced, permanently inflating + # tracked depth and wrongly denying an unrelated, genuinely + # top-level static reassignment later in the command. Confirmed live + # via a stand-in `uv`/`gh` binary on PATH that each genuinely runs + # the harmless, cleared value. + ( + "TOOL=uv; VERB=inst; VERB+=all; case 1 in (1) true ;; esac; VERB=safe; $TOOL $VERB foo", + "real-top-level-static-clear-after-a-harmless-bare-decorated-case-stays-allowed", + ), + ( + "TOOL=uv; VERB=inst; VERB+=all; case 2 in (1|2) true ;; esac; VERB=safe; $TOOL $VERB foo", + "real-top-level-static-clear-after-an-alternation-decorated-case-stays-allowed", + ), ] # --- Known, disclosed, unresolved regex/token-gate bypasses ---------------- @@ -1450,6 +1470,10 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "case A in x) case B in y) VERB=safe;; esac ;; esac | cat; $TOOL $VERB foo", "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-nested-case-whose-outer-is-piped", ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in (1) VERB=safe ;; esac & wait; $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-backgrounded-decorated-case", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index 87837461..f2fd8627 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2731,6 +2731,34 @@ def test_raw_segments_with_boundaries_a_real_subshell_nested_inside_a_case_arm_s assert depths_by_segment == [0, 1, 0, 0] +def test_raw_segments_with_boundaries_a_case_patterns_leading_decorator_paren_does_not_inflate_depth() -> None: + """FALSE-POSITIVE regression pin (round-37 independent review, issue + #1375): bash's `case` syntax allows an OPTIONAL leading `(` + decorator on a pattern arm (`(1) cmd ;;`) -- lexically identical to + a real subshell opener, but must NOT increment depth, or the SAME + round-36 fix that correctly skips decrementing depth for the arm's + own matching `)` leaves that phantom `+1` permanently unbalanced.""" + tokens = ["case", "1", "in", "(", "1", ")", "true", ";", ";", "esac", ";", "VERB=safe"] + result = checker._raw_segments_with_boundaries(tokens) + assert result[0] == (["case", "1", "in"], 0, "(") + assert result[1] == (["1"], 0, ")") + assert result[-1] == (["VERB=safe"], 0, None) + + +def test_raw_segments_with_boundaries_a_decorated_case_pattern_stays_isolating_when_backgrounded() -> None: + """No over-correction: the decorator-paren fix must not disable the + round-35/36 group-isolation mechanism itself -- a decorated case + that is genuinely backgrounded or piped still needs its own segments + to carry depth 0 here (isolation for that case comes from + `_segment_indices_isolated_by_a_piped_or_backgrounded_compound_ + group`, a separate mechanism, not from this function's own depth + field).""" + tokens = ["case", "1", "in", "(", "1", ")", "VERB=safe", ";", ";", "esac"] + result = checker._raw_segments_with_boundaries(tokens) + depths_by_segment = [depth for seg, depth, _term in result if seg] + assert depths_by_segment == [0, 0, 0, 0] + + def test_raw_segments_with_boundaries_a_nested_for_in_inside_a_case_arm_does_not_desync_pattern_tracking() -> None: """SAFETY regression pin (round-36 own design review): only the case's own FIRST `in` (right after `case WORD`) may arm the @@ -3196,6 +3224,34 @@ def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_case_in_s assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} +def test_names_reassigned_from_a_static_value_allows_a_real_top_level_clear_after_a_harmless_decorated_case() -> None: + """FALSE-POSITIVE regression pin (round-37 independent review, issue + #1375): a case pattern's OPTIONAL leading `(` decorator (`(1) ...`) + must not inflate `(...)`-nesting depth -- an EARLIER, harmless + (bare, not backgrounded or piped) decorated case must not block a + LATER, genuine top-level static reassignment from clearing + poisoning normally.""" + tokens = [ + "VERB=harmless", + ";", + "VERB=$(echo install)", + ";", + "case", + "1", + "in", + "(", + "1", + ")", + "true", + ";", + ";", + "esac", + ";", + "VERB=safe", + ] + assert checker._names_reassigned_from_a_static_value(tokens) == set() + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, isolated_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model_for_a_backgrounded_brace_group_clear_attempt( @@ -3475,6 +3531,17 @@ def test_names_cleared_by_a_later_static_reassignment_clears_after_a_harmless_ba assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} +def test_names_cleared_by_a_later_static_reassignment_clears_after_a_harmless_decorated_case() -> None: + """FALSE-POSITIVE regression pin (round-37 independent review, issue + #1375): a case pattern's OPTIONAL leading `(` decorator (`(1) ...`) + must not inflate `(...)`-nesting depth -- a bare (not backgrounded + or piped) decorated case genuinely leaks its assignment to the + parent shell in real bash, so a name reassigned static this way + must be cleared normally.""" + tokens = ["VERB=inst", "VERB+=all", ";", "case", "1", "in", "(", "1", ")", "VERB=safe", ";", ";", "esac"] + assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} + + def test_names_cleared_by_a_later_static_reassignment_clears_after_a_printf_v() -> None: tokens = ["printf", "-v", "VERB", "%s", "install", ";", "VERB=safe"] assert checker._names_cleared_by_a_later_static_reassignment(tokens, {"VERB"}) == {"VERB"} @@ -4295,6 +4362,42 @@ def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_backgrou assert verdict.deny is False +def test_classify_allows_a_real_top_level_static_clear_after_a_harmless_bare_decorated_case() -> None: + """FALSE-POSITIVE regression pin (round-37 independent review, issue + #1375): bash's `case` syntax allows an OPTIONAL leading `(` + decorator on a pattern arm (`(1) cmd ;;` -- common, POSIX/ksh- + compatible style) -- lexically identical to a real subshell opener, + but must not inflate `(...)`-nesting depth. Confirmed live via a + stand-in `uv` binary on PATH that this genuinely runs `uv safe + foo`, NOT `uv install foo` -- the decorated case has nothing to do + with the later, genuinely top-level clearing reassignment.""" + verdict = checker.classify("TOOL=uv; VERB=inst; VERB+=all; case 1 in (1) true ;; esac; VERB=safe; $TOOL $VERB foo") + assert verdict.deny is False + + +def test_classify_allows_an_alternation_decorated_case_pattern_after_a_real_top_level_clear() -> None: + """Same round, the `(1|2)`-alternation-pattern counterpart, and the + `_rule_gh_api_write` consumer. Confirmed live via a stand-in `gh` + binary on PATH that this genuinely runs `gh api ... -X safe`.""" + tool_verdict = checker.classify( + "TOOL=uv; VERB=inst; VERB+=all; case 2 in (1|2) true ;; esac; VERB=safe; $TOOL $VERB foo" + ) + assert tool_verdict.deny is False + gh_verdict = checker.classify("M=GE; M+=T; case 1 in (1) true ;; esac; M=safe; gh api repos/o/r/issues -X $M") + assert gh_verdict.deny is False + + +def test_classify_denies_a_decorated_case_pattern_that_is_genuinely_backgrounded() -> None: + """No over-correction: the decorator-paren fix must not disable the + round-35/36 group-isolation mechanism -- a decorated case that IS + genuinely backgrounded, or nested inside a real enclosing subshell, + must still deny exactly as before this round's fix.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in (1) VERB=safe ;; esac & wait; $TOOL $VERB foo" + ) + assert verdict.deny is True + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From f2bf8c412f63b06fb449565ba213dcc05ed9fe96 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 03:53:32 +0000 Subject: [PATCH 43/46] fix(hooks): stop a case statement's own subject word from desyncing tracking A fresh, independent adversarial review of this PR's current head (round 38) found a genuine security bypass in the round-36/37 case- tracking state machine, reopening the class of defect those rounds closed. Bash only treats case and esac as reserved words in command-starting position. The word between case and in -- the statement's own subject -- is an ordinary word position, so a literal esac (or case) there is valid, unremarkable bash (case esac in a) true ;; esac genuinely switches on the literal string "esac"). The case-tracking state machine matched purely on token text with no restriction that the token actually occupy real bash's own keyword position, so a literal esac subject immediately popped the tracking stack before the real in keyword was even reached. The pattern's own genuinely-terminating close paren then fell through to the ordinary subshell-close handling and wrongly decremented a REAL enclosing subshell's own tracked depth, deflating it relative to real bash's true nesting -- the opposite direction from round 37's over-denial, and therefore a genuine bypass. Independently reproduced live via classify() and real bash execution with stand-in uv/gh binaries on PATH: a case statement using the literal word "esac" as its own subject, genuinely nested inside a real enclosing subshell, resolved to deny=False even though real bash genuinely keeps the later reassignment isolated inside that subshell; the gh api write-detector counterpart and the case-as-its-own-subject variant both reproduce identically. Closed by requiring case and esac to additionally sit at position 0 of the current in-progress raw segment before either is recognized as live case-tracking syntax -- the exact same position-0 discipline already applied to the compound-command group-isolation mechanism's own open/close keyword matching, for the identical reason: real bash's grammar guarantees case and esac only ever start a fresh command, so a token seen after other tokens have already accumulated in the current segment can safely be treated as ordinary literal text. The in keyword is deliberately not given the same position-0 gate, since the real syntactic in normally is not segment-position-0 itself (it shares a segment with the case's own subject word) -- independently verified live that a literal in used as a case's own subject still resolves correctly through the existing once-only consumption guard, needing no separate fix. Regression tests added at every established layer: unit tests for the position-0 fix directly (both the esac-as-subject and case-as-subject shapes), a unit test at the poisoning-tracking level, classify()-level tests for both consumer functions plus the gh-api-write counterpart, and wrapper-level end-to-end pins against real stand-in uv/gh binaries. Full gate suite green: ruff check/format, mypy, the detection-logic property-coverage gate (already satisfied by existing coverage on the touched function), and no coverage regression on the classifier file (99%, identical pre-existing gap in two unrelated functions, confirmed unchanged against the pre-round-38 baseline). Refs #1375. --- hooks/gitapex_check_bash_safety.py | 59 ++++++++++- hooks/test_gitapex_check_bash_safety.py | 24 +++++ ...st_gitapex_check_bash_safety_properties.py | 97 +++++++++++++++++++ 3 files changed, 177 insertions(+), 3 deletions(-) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index def3adbb..ffcc9ac0 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -4231,7 +4231,60 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in token consumes the "first token" slot before its `(` is ever seen -- a deliberately narrower, disclosed residual (an over-denial on a rare, opt-in bash feature, never a bypass) rather than a fully - general case-pattern-syntax parser.""" + general case-pattern-syntax parser. + + CRITICAL bypass found by independent adversarial review (round 38, + issue #1375) and independently reproduced live, both via + `classify()` and via real bash execution with a stand-in `uv`/`gh` + binary on PATH: the round-36/37 `case`/`esac` recognition above + matched purely on TOKEN TEXT, with no restriction that the token + actually occupy real bash's own syntactic keyword position -- but + `case`/`esac` are only reserved words in COMMAND-starting position; + the word between `case` and `in` (the statement's own SUBJECT) is + an ordinary word position where a literal `esac` (or `case`) is + valid, unremarkable bash (`case esac in a) true ;; esac` genuinely + switches on the literal string `"esac"`, confirmed live via `bash + -n`). `( case esac in a) true ;; esac; VERB=safe )` -- with a + GENUINE, real enclosing subshell -- resolved to `deny=False` even + though real bash genuinely keeps `VERB=safe` isolated inside that + subshell (confirmed live via a stand-in `uv` binary on PATH: + `TOOL=uv; VERB=harmless; VERB=$(echo install); ( case esac in a) + true ;; esac; VERB=safe ); $TOOL $VERB foo` captures `uv called + with: install foo`) -- because the literal `esac` SUBJECT token + immediately popped the case-tracking stack frame the real `case` + keyword one token earlier had just pushed, before the real `in` was + even reached; the pattern's own genuinely-terminating `)` (after + `a`) then fell through to the ordinary subshell-close branch and + wrongly decremented the REAL enclosing subshell's own tracked + depth, deflating it relative to real bash's true nesting -- the + opposite direction from round 37's over-denial, and therefore a + genuine bypass, not a false positive. `_rule_gh_api_write` + reproduces identically (`M=safe; M=$(echo POST); ( case esac in a) + true ;; esac; M=GET ); gh api repos/o/r/pulls/1/merge -X $M` -- + real bash genuinely issues an unreviewed `-X POST` write). + + Closed by requiring `case`/`esac` to additionally sit at position 0 + of the CURRENT in-progress raw segment (`not segments[-1]` at the + moment each is seen, before it is itself appended) -- the exact + same position-0 discipline `_segment_indices_isolated_by_a_piped_ + or_backgrounded_compound_group` already applies to its own + `_GROUP_OPEN_KEYWORDS`/`_GROUP_CLOSE_KEYWORDS` matching, for the + identical reason: real bash's grammar guarantees `case`/`esac` only + ever start a fresh command, so a token seen after other tokens have + already accumulated in the current segment (e.g. immediately after + `case`, in subject position) can safely be treated as ordinary + literal text. `in` is deliberately NOT given the same position-0 + gate -- unlike `case`/`esac`, the real syntactic `in` keyword is + normally NOT segment-position-0 (it shares a segment with the + case's own subject word, e.g. `["case", "1", "in"]`), so a + position-0 requirement would break the legitimate case instead of + only the collision; `in`'s own existing `case_seen_in`-gated + once-only consumption was independently verified live to still + resolve `case in in a) ...`'s own literal-`in`-as-subject collision + correctly (both interpretations of which `in` is "the real one" + coincidentally land on the same correct pattern-close outcome for + the arm's own terminating `)`), so no separate fix was needed + there.""" segments: list[list[str]] = [[]] seg_depths: list[int] = [0] terminators: list[str | None] = [] @@ -4242,7 +4295,7 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in prev_tok: str | None = None for tok in tokens: is_case_setup_token = False - if tok == "case": + if tok == "case" and not segments[-1]: case_awaiting_pattern_close.append(False) case_seen_in.append(False) case_pattern_started.append(False) @@ -4252,7 +4305,7 @@ def _raw_segments_with_boundaries(tokens: list[str]) -> list[tuple[list[str], in case_awaiting_pattern_close[-1] = True case_pattern_started[-1] = False is_case_setup_token = True - elif tok == "esac" and case_awaiting_pattern_close: + elif tok == "esac" and case_awaiting_pattern_close and not segments[-1]: case_awaiting_pattern_close.pop() case_seen_in.pop() case_pattern_started.pop() diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index bbbb5cb2..afc27c15 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -1474,6 +1474,30 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); case 1 in (1) VERB=safe ;; esac & wait; $TOOL $VERB foo", "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-backgrounded-decorated-case", ), + # CRITICAL bypass found live by Step 8 independent review, + # thirty-eighth round (issue #1375): `case`/`esac` are only reserved + # words in COMMAND-starting position -- the case statement's own + # SUBJECT word (between `case` and `in`) is an ordinary word + # position where a literal `esac` (or `case`) is valid, unremarkable + # bash. The round-36/37 case-tracking state machine matched purely + # on token text with no position check, so a literal `esac` subject + # immediately popped the tracking stack before the real `in` was + # even reached, corrupting a GENUINELY enclosing real subshell's own + # tracked depth. Confirmed live via a stand-in `uv`/`gh` binary on + # PATH that each genuinely keeps the reassignment isolated inside + # the real subshell. + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); ( case esac in a) true ;; esac; VERB=safe ); $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-case-subject-word-matching-esac", + ), + ( + "M=safe; M=$(echo POST); ( case esac in a) true ;; esac; M=GET ); gh api repos/o/r/pulls/1/merge -X $M", + "gh-api-method-value-reassigned-from-a-static-value-cleared-via-a-case-subject-word-matching-esac", + ), + ( + "TOOL=uv; VERB=harmless; VERB=$(echo install); ( case case in a) true ;; esac; VERB=safe ); $TOOL $VERB foo", + "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-case-subject-word-matching-case", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index f2fd8627..cd9cee84 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -2759,6 +2759,33 @@ def test_raw_segments_with_boundaries_a_decorated_case_pattern_stays_isolating_w assert depths_by_segment == [0, 0, 0, 0] +def test_raw_segments_with_boundaries_a_case_subject_word_matching_esac_does_not_desync_pattern_tracking() -> None: + """CRITICAL bypass regression pin (round-38 independent review, issue + #1375): `case`/`esac` are only reserved words in COMMAND-starting + position -- the case statement's own SUBJECT word (between `case` + and `in`) is an ordinary word position where a literal `esac` is + valid, unremarkable bash. A literal `esac` subject must not be + mistaken for the real closing keyword, or the real enclosing + subshell's own tracked depth gets prematurely decremented on the + arm's own genuine pattern-terminating `)`.""" + tokens = ["(", "case", "esac", "in", "a", ")", "true", ";", ";", "esac", ";", "VERB=safe", ")"] + result = checker._raw_segments_with_boundaries(tokens) + assert result[0] == ([], 0, "(") + assert result[1] == (["case", "esac", "in", "a"], 1, ")") + assert result[-2] == (["VERB=safe"], 1, ")") + assert result[-1] == ([], 0, None) + + +def test_raw_segments_with_boundaries_a_case_subject_word_matching_case_does_not_desync_pattern_tracking() -> None: + """Same round, the `case`-as-its-own-subject counterpart -- a + literal `case` subject must not spuriously push a SECOND, + unmatched case-tracking stack frame.""" + tokens = ["(", "case", "case", "in", "a", ")", "true", ";", ";", "esac", ";", "VERB=safe", ")"] + result = checker._raw_segments_with_boundaries(tokens) + assert result[1] == (["case", "case", "in", "a"], 1, ")") + assert result[-2] == (["VERB=safe"], 1, ")") + + def test_raw_segments_with_boundaries_a_nested_for_in_inside_a_case_arm_does_not_desync_pattern_tracking() -> None: """SAFETY regression pin (round-36 own design review): only the case's own FIRST `in` (right after `case WORD`) may arm the @@ -3252,6 +3279,36 @@ def test_names_reassigned_from_a_static_value_allows_a_real_top_level_clear_afte assert checker._names_reassigned_from_a_static_value(tokens) == set() +def test_names_reassigned_from_a_static_value_stays_poisoned_despite_a_case_subject_word_matching_esac() -> None: + """CRITICAL bypass regression pin (round-38 independent review, issue + #1375): a literal `esac` used as a case statement's own SUBJECT + word (an ordinary word position, not the reserved-word position) + must not be mistaken for the real closing keyword -- a static + reassignment inside a GENUINELY enclosing subshell around such a + case must still stay poisoned even with no `&`/`|` involved at + all.""" + tokens = [ + "VERB=harmless", + ";", + "VERB=$(echo install)", + ";", + "(", + "case", + "esac", + "in", + "a", + ")", + "true", + ";", + ";", + "esac", + ";", + "VERB=safe", + ")", + ] + assert checker._names_reassigned_from_a_static_value(tokens) == {"VERB"} + + @_PROPERTIES @given(name=_IDENTIFIERS, static_value=_VALUES, dynamic_value=_VALUES, isolated_value=_VALUES) def test_names_reassigned_from_a_static_value_matches_model_for_a_backgrounded_brace_group_clear_attempt( @@ -4398,6 +4455,46 @@ def test_classify_denies_a_decorated_case_pattern_that_is_genuinely_backgrounded assert verdict.deny is True +def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_case_subject_word_matching_esac() -> ( + None +): + """CRITICAL bypass regression pin (round-38 independent review, + issue #1375): `case`/`esac` are only reserved words in COMMAND- + starting position -- the case statement's own SUBJECT word (between + `case` and `in`) is an ordinary word position where a literal + `esac` is valid, unremarkable bash (`case esac in a) true ;; esac` + genuinely switches on the literal string "esac", confirmed live via + `bash -n`). Confirmed live via a stand-in `uv` binary on PATH that + the GENUINE enclosing subshell here still isolates `VERB=safe` in + real bash, so this must deny -- captured argv `uv called with: + install foo`, NOT `safe foo`.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); ( case esac in a) true ;; esac; VERB=safe ); $TOOL $VERB foo" + ) + assert verdict.deny is True + + +def test_classify_denies_a_gh_api_method_reassigned_from_a_static_value_via_a_case_subject_word_matching_esac() -> None: + """Companion to the B1b pin above, for `_rule_gh_api_write`. + Confirmed live via a stand-in `gh` binary on PATH that this + genuinely runs `gh api repos/o/r/pulls/1/merge -X POST`, a genuine + unreviewed write.""" + verdict = checker.classify( + "M=safe; M=$(echo POST); ( case esac in a) true ;; esac; M=GET ); gh api repos/o/r/pulls/1/merge -X $M" + ) + assert verdict.deny is True + + +def test_classify_denies_a_case_subject_word_matching_case_itself() -> None: + """Same round, the `case`-as-its-own-subject counterpart -- a + literal `case` subject must not spuriously push a second, unmatched + case-tracking stack frame.""" + verdict = checker.classify( + "TOOL=uv; VERB=harmless; VERB=$(echo install); ( case case in a) true ;; esac; VERB=safe ); $TOOL $VERB foo" + ) + assert verdict.deny is True + + def test_resolve_path_tokens_denies_a_name_with_a_dynamic_assignment_elsewhere() -> None: """End-to-end regression pin for the round-24 finding at the `_resolve_path_tokens` level. Confirmed live before this fix: `DIR= From aeff40afe92bece74a09e27a8db6e9b9f2465267 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 04:17:08 +0000 Subject: [PATCH 44/46] docs(hooks): disclose a critical quoted-paren subshell-depth-tracking bypass A fresh, independent adversarial review of this PR's current head (round 39), stress-testing rounds 30-38's own scope-isolation reassignment-clearing story, found a third instance of the same shlex-quote-information-loss class already disclosed as issue #1404 (nested double-quote state) and issue #1412 (redirect-operator-shaped filenames). _raw_segments_with_boundaries (the (...)-subshell depth tracker underlying _names_reassigned_from_a_static_value/_names_cleared_by_a_ later_static_reassignment, and by extension _rule_gh_api_write/B1a/B1b) recognizes a real subshell open/close purely by a token's TEXT -- tokenize()'s own shlex dequotes every token first, so a QUOTED literal "(" or ")" argument tokenizes to the identical bare string as a genuine, unquoted operator, with no way to recover which one the source actually was. Two distinct manifestations confirmed live: a quoted close paren inside a genuinely enclosing subshell prematurely decrements tracked depth, wrongly letting a still-isolated reassignment clear an earlier poisoning -- real bash genuinely keeps the reassignment isolated, but classify() allows the command outright; and a quoted open paren with no matching close inflates depth for the rest of the command with nothing to ever balance it, wrongly denying an ordinary, harmless, genuinely top-level clearing reassignment that follows. The first is a genuine security bypass; the second is a false positive -- both confirmed via classify() against the live module source and via real bash execution with stand-in uv/gh binaries on PATH, and reproducing identically for the gh api write-detector counterpart. More severe in reach than the two already-disclosed residuals, since the mechanism it defeats is the one every round-30-38 finding exists to protect. Deliberately not attempted here, for the identical reason issue #1412 already gives: a narrow patch confined to this one function risks reintroducing a worse, far more common false-positive class, and a genuinely sound fix needs tokenize() itself to preserve per-token quote/escape provenance -- the same tokenizer-level architectural change issues #1404/#1412 already require, ideally landed once for all three rather than three independently-drifting patches. Tracked as its own dedicated issue, matching this module's own established convention for the prior two instances of this class: https://github.com/tvna/gitapex/issues/1502 Disclosed in the module's own header docstring and pinned as tested, current-behavior regressions: the bypass direction in KNOWN_BYPASS_COMMANDS (quoted-paren-inside-a-subshell-clears-a- poisoning-bypass, plus the gh-api-write counterpart), and the false-positive direction as a disclosed over-denial residual test, mirroring the existing arithmetic-double-paren-content precedent. Refs #1375. --- hooks/gitapex_check_bash_safety.py | 43 ++++++++++++++++++ hooks/test_gitapex_check_bash_safety.py | 45 +++++++++++++++++++ ...st_gitapex_check_bash_safety_properties.py | 29 ++++++++++++ 3 files changed, 117 insertions(+) diff --git a/hooks/gitapex_check_bash_safety.py b/hooks/gitapex_check_bash_safety.py index ffcc9ac0..1043b0dd 100644 --- a/hooks/gitapex_check_bash_safety.py +++ b/hooks/gitapex_check_bash_safety.py @@ -141,6 +141,49 @@ pinned as `quoted-redirect-operator-shaped-filename-bypass` in hooks/test_gitapex_check_bash_safety.py's own `KNOWN_BYPASS_COMMANDS`. +CRITICAL, disclosed, third instance of the SAME shlex-quote-information- +loss class as #1404/#1412 above (found live by Step 8 independent +review, round 39 of issue #1375's own checkout/restore feature review, +while stress-testing rounds 30-38's own scope-isolation reassignment- +clearing story): `_raw_segments_with_boundaries` (the `(...)`-subshell +depth tracker underlying `_names_reassigned_from_a_static_value`/ +`_names_cleared_by_a_later_static_reassignment`, and by extension +`_rule_gh_api_write`/B1a/B1b) recognizes a real subshell open/close +purely by a token's TEXT (`tok in _SUBSHELL_OPEN_TOKENS`, `tok == ")"`) +-- but `tokenize()`'s own shlex dequoting makes a QUOTED literal `"("`/ +`")"` argument tokenize to the identical bare string as a genuine, +unquoted operator, with no way to recover which one the source actually +was, exactly the same information loss #1404/#1412 already document for +different consumers. TWO distinct manifestations confirmed live: a +quoted `)` inside a genuinely enclosing `(...)` subshell (`TOOL=uv; +VERB=harmless; VERB=$(echo install); ( true ")" ; VERB=safe ); $TOOL +$VERB foo`) prematurely decrements tracked depth, wrongly letting the +still-isolated `VERB=safe` clear an earlier poisoning -- confirmed live +via a stand-in `uv` binary on PATH that real bash genuinely still runs +`uv install foo`, NOT `safe foo` (the `_rule_gh_api_write` counterpart +reproduces identically, a genuine bypass, not a false positive); and a +quoted `(` with no matching close (`echo "("`) inflates depth for the +rest of the command with nothing to ever balance it, wrongly denying an +ordinary, harmless, genuinely top-level clearing reassignment that +follows -- confirmed live via a stand-in `uv` binary that real bash +genuinely runs the harmless command, a false positive this time. More +severe in reach than #1404/#1412, since the mechanism it defeats is the +one every round-30-38 finding exists to protect. Deliberately NOT +attempted here, for the identical reason #1412 already gives: a narrow +patch confined to this one function risks reintroducing a worse, +far-more-common false-positive class, and a genuinely sound fix needs +`tokenize()` itself to preserve per-token quote/escape provenance -- +the same tokenizer-level architectural change #1404/#1412 already +require, ideally landed once for all three rather than three +independently-drifting patches. Tracked as its own dedicated issue, +https://github.com/tvna/gitapex/issues/1502; pinned as +`quoted-paren-inside-a-subshell-clears-a-poisoning-bypass`/ +`gh-api-quoted-paren-inside-a-subshell-clears-a-poisoning-bypass` in +hooks/test_gitapex_check_bash_safety.py's own `KNOWN_BYPASS_COMMANDS` +(the bypass direction), and as a disclosed over-denial residual (the +false-positive direction) alongside this module's own other accepted +over-denials. + A second, distinct, round-17 finding in the SAME redirect-handling area is NOT a bypass and is NOT tracked separately: `_redirect_span_length`'s own deliberate choice (round 16) to leave a leading digit token OUT of diff --git a/hooks/test_gitapex_check_bash_safety.py b/hooks/test_gitapex_check_bash_safety.py index afc27c15..80e03dfe 100644 --- a/hooks/test_gitapex_check_bash_safety.py +++ b/hooks/test_gitapex_check_bash_safety.py @@ -707,6 +707,38 @@ def assert_allowed(command: str) -> None: 'git checkout ">" realfile.py', "quoted-redirect-operator-shaped-filename-bypass", ), + ( + # CRITICAL bypass, third instance of the same underlying + # shlex-quote-information-loss class as the two residuals above. + # Found live by independent adversarial review (round 39, issue + # #1375), tracked as its own dedicated issue rather than fixed + # here: https://github.com/tvna/gitapex/issues/1502 -- + # deliberately out of issue #1375's own scope, for the identical + # reason issue #1412 already gives: a narrow patch confined to + # `_raw_segments_with_boundaries` alone risks reintroducing a + # worse, far more common false-positive class; a genuine fix + # needs tokenize() itself to preserve per-token quote/escape + # provenance, the same class of tokenizer-level change issues + # #1404/#1412 already require. `_raw_segments_with_boundaries` + # recognizes a real subshell close purely by a token's final TEXT + # value (`tok == ")"`) -- `tokenize()`'s own shlex dequotes every + # token first, so a QUOTED `")"` argument inside a genuinely + # enclosing `(...)` subshell tokenizes identically to a real, + # unquoted subshell-closing paren, prematurely decrementing + # tracked depth. Live-verified: real bash genuinely still runs + # `uv install foo` (`VERB=safe` never escapes the real subshell), + # while `classify()` reports `deny=False`. See issue #1502 for + # the full write-up, the companion false-positive shape, and + # live-verification detail. + 'TOOL=uv; VERB=harmless; VERB=$(echo install); ( true ")" ; VERB=safe ); $TOOL $VERB foo', + "quoted-paren-inside-a-subshell-clears-a-poisoning-bypass", + ), + ( + # Companion to the bypass just above, for `_rule_gh_api_write`. + # See issue #1502. + 'M=safe; M=$(echo POST); ( true ")" ; M=GET ); gh api repos/o/r/pulls/1/merge -X $M', + "gh-api-quoted-paren-inside-a-subshell-clears-a-poisoning-bypass", + ), ] @@ -1498,6 +1530,19 @@ def test_known_bypass_still_unblocked(command: str, case_id: str) -> None: "TOOL=uv; VERB=harmless; VERB=$(echo install); ( case case in a) true ;; esac; VERB=safe ); $TOOL $VERB foo", "var-split-tool-and-verb-reassigned-from-a-static-value-cleared-via-a-case-subject-word-matching-case", ), + # Same round (39), the disclosed quoted-open-paren over-denial + # residual (issue #1502): a quoted "(" with no matching close + # inflates tracked subshell depth for the rest of the command, + # wrongly denying an ordinary, harmless, genuinely top-level + # clearing reassignment that follows. See `_raw_segments_with_ + # boundaries`'s own module-docstring disclosure and the companion + # `quoted-paren-inside-a-subshell-clears-a-poisoning-bypass` entry + # above (the bypass direction of the same shlex-quote-information- + # loss class). + ( + 'TOOL=uv; VERB=harmless; VERB=$(echo install); echo "("; VERB=safe; $TOOL $VERB foo', + "quoted-open-paren-inflates-depth-stays-denied-as-a-disclosed-residual", + ), ] diff --git a/tests/test_gitapex_check_bash_safety_properties.py b/tests/test_gitapex_check_bash_safety_properties.py index cd9cee84..275e8c6d 100644 --- a/tests/test_gitapex_check_bash_safety_properties.py +++ b/tests/test_gitapex_check_bash_safety_properties.py @@ -4020,6 +4020,35 @@ def test_classify_denies_a_deliberately_spaced_double_subshell_distractor() -> N assert verdict.deny is True +def test_classify_allows_a_quoted_open_paren_as_a_disclosed_over_denial_residual() -> None: + """DISCLOSED, deliberately NOT fixed (round 39 independent review, + issue #1375, tracked as https://github.com/tvna/gitapex/issues/1502): + `_raw_segments_with_boundaries` recognizes a real subshell opener + purely by a token's TEXT (`tok in _SUBSHELL_OPEN_TOKENS`) -- + `tokenize()`'s own shlex dequotes every token first, so a QUOTED + `"("` argument with no matching close tokenizes identically to a + real, unquoted subshell opener, inflating tracked depth for the + REST of the command with nothing to ever balance it. Confirmed live + via a stand-in `uv` binary on PATH that real bash genuinely runs the + harmless `uv safe foo` (`echo "("` just prints a literal `(`, and + `VERB=safe` is an ordinary top-level clearing assignment) -- but + this classifier wrongly denies it. This is the FALSE-POSITIVE + direction of the same shlex quote-information-loss class as the + `quoted-paren-inside-a-subshell-clears-a-poisoning-bypass` entry in + `hooks/test_gitapex_check_bash_safety.py`'s own + `KNOWN_BYPASS_COMMANDS` (that entry pins the BYPASS direction); + fixing either soundly needs `tokenize()` itself to preserve + per-token quote/escape provenance, the same tokenizer-level change + issues #1404/#1412 already require -- deliberately not attempted + here, matching this module's own established convention for that + disclosed residual class. This is currently expected (denied) + behavior, not a should-be-fixed assertion -- if this ever starts + passing, the underlying gap closed; update this test (and issue + #1502) together.""" + verdict = checker.classify('TOOL=uv; VERB=harmless; VERB=$(echo install); echo "("; VERB=safe; $TOOL $VERB foo') + assert verdict.deny is True + + def test_classify_denies_a_b1b_tool_and_verb_reassigned_from_a_static_value_via_a_process_substitution_clear() -> None: """CRITICAL bypass regression pin (round-33 independent review, issue #1375): a process substitution runs its own content in a From 9f6025e3290eb81ff48cfcc5f7db9a0f55c73568 Mon Sep 17 00:00:00 2001 From: t Date: Sun, 30 Aug 2026 04:47:51 +0000 Subject: [PATCH 45/46] fix(hooks): recognize conflict side flags as unambiguous checkout paths A fresh, independent adversarial review of this PR's current head (round 40), branching away from the rounds 19-39 reassignment-scope- isolation story into checkout/restore path resolution itself, found a fourth class of checkout/restore protection gap: `_git_checkout_paths` only special-cases `-b`/`-B`/`--orphan` (`_CHECKOUT_BRANCH_CREATION_ FLAGS`) as genuinely ref-vs-path-ambiguous, folding a single trailing positional after one of them into the bare-SOMENAME Non-goal. But `--ours`/`--theirs`/`-2`/`-3` (git's own conflict-resolution side flags) do not share that ambiguity -- real git flatly refuses to combine them with branch switching (live-verified against real git 2.43.0: `git checkout --ours otherbranch`, with `otherbranch` a real ref, reports `fatal: '--ours/--theirs' cannot be used with switching branches`) -- so a single remaining positional after one of them is unambiguously a path, never a ref. The pre-fix code fell through to the Non-goal anyway, so `git checkout --ours realfile.py` (and `--theirs`/`-2`/`-3`) resolved to an empty `checkout_restore_paths`, skipping the wrapper's own live dirty-file check entirely -- live- verified that each of the four flags genuinely discards dirty tracked- file content in real git while classify() reported nothing to check. Also live-verified the negative control: `--conflict=