Skip to content

fix(grep): send read diagnostics to stderr and exit 2 - #2441

Merged
chaliy merged 1 commit into
mainfrom
claude/pensive-hypatia-arscn9
Sep 23, 2026
Merged

chaliy merged 1 commit into
mainfrom
claude/pensive-hypatia-arscn9

Conversation

@chaliy

@chaliy chaliy commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

What changed

grep reported an unreadable operand on the wrong stream, with the wrong
status, and silently changed the format of the lines that did match:

$ grep hello ok.txt /nope
before after (= GNU grep 3.11)
stdout grep: /nope: io error: file not found
hello
ok.txt:hello
stderr (empty) grep: /nope: No such file or directory
exit 0 2

The same bashkit::Error Display leak is also fixed in rg, sed and
json — the last three readers outside the shared helper that #2440 covered.
All thirty file-reading builtins are now clean.

Why

A diagnostic in stdout is not a cosmetic problem, it is data corruption:
grep foo *.log | wc -l counted the error line as a match. Anything
consuming grep's output — a pipeline, an agent parsing results — saw it as
content.

Four separate defects, each verified against GNU grep 3.11:

  1. Wrong stream. The message was pushed into the stdout accumulator.
  2. Wrong status. An unreadable operand must set exit 2, which outranks
    both "matched" (0) and "no match" (1). bashkit returned whichever the
    match produced.
  3. Leaked enum shape. io error: is the Display of bashkit::Error
    (TM-INF-022), the same leak as fix(hextools): report a missing operand the way real od does #2438 and fix(builtins): report unreadable operands with the real errno text #2440.
  4. Wrong filename prefix. show_filename was computed from the operands
    grep managed to read, not the ones it was asked for, so a failing
    operand silently dropped the ok.txt: prefix from the lines that matched.

Before / After

The full matrix, matching GNU grep 3.11 exactly on all three of exit status,
stdout and stderr:

command exit stdout stderr
grep hello ok.txt /nope 2 ok.txt:hello grep: /nope: No such file…
grep hello /nope 2 — grep: /nope: No such file…
grep zzz ok.txt /nope 2 — grep: /nope: No such file…
grep -s hello ok.txt /nope 2 ok.txt:hello —
grep -s zzz ok.txt /nope 2 — —
grep -q hello ok.txt /nope 0 — —
grep -q zzz ok.txt /nope 2 — grep: /nope: No such file…
grep hello ok.txt 0 hello —

Two flag interactions worth calling out, because they are not symmetric:

  • -s silences the message but not the status — it is --no-messages,
    not --no-errors.
  • -q stops at the first match, so it never reaches a later unreadable
    operand: status 0, nothing on stderr. Without a match it does reach it and
    reports normally. bashkit reads every operand up front, so the short-circuit
    is emulated rather than inherited.

And the remaining three leaks:

rg foo /nope       rg: /nope: io error: file not found       -> rg: /nope: No such file or directory
sed s/a/b/ /nope   sed: can't read /nope: io error: …        -> sed: can't read /nope: No such file or directory
json get x /nope   json: /nope: io error: file not found     -> json: /nope: No such file or directory

Risk

  • Medium — this is the first change in the series that alters exit codes
    and stream routing, not just message text.
  • A caller that treated grep's missing-file case as exit 0/1 now sees 2.
    That is the documented GNU behaviour and the reason for the change, but it
    is a real behavioural difference.
  • The -q short-circuit is emulated, not inherited: bashkit reads all
    operands before matching, so the observable effect is reproduced by dropping
    the message and status when -q found a match. That matches GNU grep for
    every case in the table above.
  • rg, sed and json are text-only changes, no status or stream change.

Checklist

  • Tests added or updated
  • Backward compatibility considered

Tests: a grep_unreadable_operand module covering all six behaviours
separately — stream routing, the filename prefix, status 2 outranking both
match outcomes, -s keeping the status, the -q asymmetry, and a
grep … | wc -l test pinning the pipeline bug that motivated the split.
builtin_fs_error_tests now walks all thirty file-reading builtins,
asserting the real errno text, each tool's own exit code, the absence of both
banned shapes in either stream, and the full assert_no_leak list.

Three pre-existing grep unit tests asserted the old behaviour
(exit_code == 1, and the message found in result.stdout). They are updated
to the GNU values, which strengthens them rather than relaxing them.

One spec expectation was wrong and is corrected.
spec_cases/grep/grep.test.sh:grep_suppress_errors expected exit 1 for
grep -s foo /tmp/nonexistent; real grep exits 2. The expectation had
captured bashkit's old behaviour. That file is not in the host-comparison set,
which is why it survived — worth knowing, since the same blind spot could hide
other coreutils spec drift.

Validated: cargo fmt --check, cargo clippy --workspace --all-targets --features http_client,ssh,sqlite -- -D warnings, the CI test slices (3484 +
1827 + 168 doc), the jq/http_client slice for the rg/json/sed
changes, proptest_security, security_failpoint_tests, 95/95 grep spec
tests, check-bash-compat ("All non-excluded tests match real bash"),
check-okf, check-doc-links. ssh_supabase_tests cannot run here — no
port-22 egress in this sandbox, the same condition CI skips the live SSH test
under.

Still open after this

The shared helper labels the resolved path, not the operand: in /work,
wc nope reports wc: /work/nope: where real wc echoes wc: nope:.
Fixing it means threading the operand through 33 call sites, so it is left
out of this change. cat, head, tail and grep already label correctly.


Generated by Claude Code

`grep hello ok.txt /nope` wrote `grep: /nope: io error: file not found` into
**stdout**, so the diagnostic became part of the data a pipeline consumes:
`grep foo *.log | wc -l` counted it as a match. It also exited 0, and printed
the matching line without its `ok.txt:` prefix.

Four defects, all checked against GNU grep 3.11:

1. the diagnostic went to stdout — it now goes to stderr;
2. an unreadable operand did not set status 2, which outranks both "matched"
   and "no match". `-s` silences the message but not the status; `-q` stops at
   the first match, so a match there still exits 0 and prints nothing, while
   `-q` without a match does reach the bad operand and reports it;
3. the reason was the `Display` of `bashkit::Error`;
4. the filename prefix followed the operands grep managed to read rather than
   the operands it was asked for, so a failed operand silently changed the
   format of the lines that did match.

The same `Display` leak is also fixed in `rg`, `sed` and `json`, the last
three readers outside the shared helper that #2440 covered. All thirty
file-reading builtins are now clean.

`spec_cases/grep/grep.test.sh:grep_suppress_errors` expected exit 1 where GNU
grep exits 2; it had captured the old bashkit behaviour. That file is not in
the host-comparison set, which is why the wrong expectation survived.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
bashkit e20f7bd Commit Preview URL

Branch Preview URL
Sep 23 2026, 10:40 AM

@chaliy
chaliy merged commit 94adf4c into main Sep 23, 2026
44 checks passed
@chaliy
chaliy deleted the claude/pensive-hypatia-arscn9 branch September 23, 2026 10:54
chaliy added a commit that referenced this pull request Sep 23, 2026
)

The grep spec suite had never been compared against the real tool, so its expectations recorded whatever Bashkit printed when they were written. Pointing the differential gate at it found six divergences from GNU grep 3.11, every one of them encoded in the spec as if correct:

    printf 'foo\n' | grep -l foo          (stdin)        -> (standard input)
    printf 'foo\0bar\0' | grep -z foo     foo\n          -> foo\0
    printf 'foo\0bar\n' | grep -a foo     foobar         -> foo\0bar
    printf 'foo\0bar\n' | grep foo        stdout notice  -> stderr, GNU wording
    grep -L foo matching.txt              exit 1         -> exit 0
    grep -L foo other.txt                 exit 0         -> exit 1

Two are more than cosmetic. `-a` means "treat this binary file as text", not "delete its NUL bytes", so stripping them altered the bytes the caller got back. The binary-match notice is a diagnostic, so GNU >= 3.5 writes `grep: FILE: binary file matches` to stderr; on stdout it joined the data a pipeline reads, the same way the read diagnostic did before #2441.

`-L`'s status was inverted. GNU reports whether a *match* was found, whatever `-L` chose to print: a matching file prints nothing and exits 0, a non-matching file prints its name and exits 1.

`--color=always` stays a no-op and is now marked `### bash_diff` rather than asserting plain output as if it were GNU's.

The gate itself is the point. `grep_comparison_tests` joins the bash and sed gates, and CI now runs all three (`spec_tests:: --ignored`) instead of only the bash one — which is why the sed gate added with the GNU-sed work had never actually run there either. Spec suite 95/95; the gate compares 94/94 against the host, the one exclusion being the `bash_diff`-marked colour case.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant