Repository navigation
fix(grep): send read diagnostics to stderr and exit 2 - #2441
Merged
Merged
Conversation
`grep hello ok.txt /nope` wrote `grep: /nope: io error: file not found` into **stdout**, so the diagnostic became part of the data a pipeline consumes: `grep foo *.log | wc -l` counted it as a match. It also exited 0, and printed the matching line without its `ok.txt:` prefix. Four defects, all checked against GNU grep 3.11: 1. the diagnostic went to stdout — it now goes to stderr; 2. an unreadable operand did not set status 2, which outranks both "matched" and "no match". `-s` silences the message but not the status; `-q` stops at the first match, so a match there still exits 0 and prints nothing, while `-q` without a match does reach the bad operand and reports it; 3. the reason was the `Display` of `bashkit::Error`; 4. the filename prefix followed the operands grep managed to read rather than the operands it was asked for, so a failed operand silently changed the format of the lines that did match. The same `Display` leak is also fixed in `rg`, `sed` and `json`, the last three readers outside the shared helper that #2440 covered. All thirty file-reading builtins are now clean. `spec_cases/grep/grep.test.sh:grep_suppress_errors` expected exit 1 where GNU grep exits 2; it had captured the old bashkit behaviour. That file is not in the host-comparison set, which is why the wrong expectation survived.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
bashkit | e20f7bd | Commit Preview URL Branch Preview URL |
Sep 23 2026, 10:40 AM |
2 tasks done
chaliy
added a commit
that referenced
this pull request
Sep 23, 2026
) The grep spec suite had never been compared against the real tool, so its expectations recorded whatever Bashkit printed when they were written. Pointing the differential gate at it found six divergences from GNU grep 3.11, every one of them encoded in the spec as if correct: printf 'foo\n' | grep -l foo (stdin) -> (standard input) printf 'foo\0bar\0' | grep -z foo foo\n -> foo\0 printf 'foo\0bar\n' | grep -a foo foobar -> foo\0bar printf 'foo\0bar\n' | grep foo stdout notice -> stderr, GNU wording grep -L foo matching.txt exit 1 -> exit 0 grep -L foo other.txt exit 0 -> exit 1 Two are more than cosmetic. `-a` means "treat this binary file as text", not "delete its NUL bytes", so stripping them altered the bytes the caller got back. The binary-match notice is a diagnostic, so GNU >= 3.5 writes `grep: FILE: binary file matches` to stderr; on stdout it joined the data a pipeline reads, the same way the read diagnostic did before #2441. `-L`'s status was inverted. GNU reports whether a *match* was found, whatever `-L` chose to print: a matching file prints nothing and exits 0, a non-matching file prints its name and exits 1. `--color=always` stays a no-op and is now marked `### bash_diff` rather than asserting plain output as if it were GNU's. The gate itself is the point. `grep_comparison_tests` joins the bash and sed gates, and CI now runs all three (`spec_tests:: --ignored`) instead of only the bash one — which is why the sed gate added with the GNU-sed work had never actually run there either. Spec suite 95/95; the gate compares 94/94 against the host, the one exclusion being the `bash_diff`-marked colour case.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
grepreported an unreadable operand on the wrong stream, with the wrongstatus, and silently changed the format of the lines that did match:
grep: /nope: io error: file not foundhellook.txt:hellogrep: /nope: No such file or directory02The same
bashkit::ErrorDisplayleak is also fixed inrg,sedandjson— the last three readers outside the shared helper that #2440 covered.All thirty file-reading builtins are now clean.
Why
A diagnostic in stdout is not a cosmetic problem, it is data corruption:
grep foo *.log | wc -lcounted the error line as a match. Anythingconsuming grep's output — a pipeline, an agent parsing results — saw it as
content.
Four separate defects, each verified against GNU grep 3.11:
both "matched" (0) and "no match" (1). bashkit returned whichever the
match produced.
io error:is theDisplayofbashkit::Error(TM-INF-022), the same leak as fix(hextools): report a missing operand the way real od does #2438 and fix(builtins): report unreadable operands with the real errno text #2440.
show_filenamewas computed from the operandsgrep managed to read, not the ones it was asked for, so a failing
operand silently dropped the
ok.txt:prefix from the lines that matched.Before / After
The full matrix, matching GNU grep 3.11 exactly on all three of exit status,
stdout and stderr:
grep hello ok.txt /nopeok.txt:hellogrep: /nope: No such file…grep hello /nopegrep: /nope: No such file…grep zzz ok.txt /nopegrep: /nope: No such file…grep -s hello ok.txt /nopeok.txt:hellogrep -s zzz ok.txt /nopegrep -q hello ok.txt /nopegrep -q zzz ok.txt /nopegrep: /nope: No such file…grep hello ok.txthelloTwo flag interactions worth calling out, because they are not symmetric:
-ssilences the message but not the status — it is--no-messages,not
--no-errors.-qstops at the first match, so it never reaches a later unreadableoperand: status 0, nothing on stderr. Without a match it does reach it and
reports normally. bashkit reads every operand up front, so the short-circuit
is emulated rather than inherited.
And the remaining three leaks:
Risk
and stream routing, not just message text.
grep's missing-file case as exit 0/1 now sees 2.That is the documented GNU behaviour and the reason for the change, but it
is a real behavioural difference.
-qshort-circuit is emulated, not inherited: bashkit reads alloperands before matching, so the observable effect is reproduced by dropping
the message and status when
-qfound a match. That matches GNU grep forevery case in the table above.
rg,sedandjsonare text-only changes, no status or stream change.Checklist
Tests: a
grep_unreadable_operandmodule covering all six behavioursseparately — stream routing, the filename prefix, status 2 outranking both
match outcomes,
-skeeping the status, the-qasymmetry, and agrep … | wc -ltest pinning the pipeline bug that motivated the split.builtin_fs_error_testsnow walks all thirty file-reading builtins,asserting the real errno text, each tool's own exit code, the absence of both
banned shapes in either stream, and the full
assert_no_leaklist.Three pre-existing
grepunit tests asserted the old behaviour(
exit_code == 1, and the message found inresult.stdout). They are updatedto the GNU values, which strengthens them rather than relaxing them.
One spec expectation was wrong and is corrected.
spec_cases/grep/grep.test.sh:grep_suppress_errorsexpected exit 1 forgrep -s foo /tmp/nonexistent; real grep exits 2. The expectation hadcaptured bashkit's old behaviour. That file is not in the host-comparison set,
which is why it survived — worth knowing, since the same blind spot could hide
other coreutils spec drift.
Validated:
cargo fmt --check,cargo clippy --workspace --all-targets --features http_client,ssh,sqlite -- -D warnings, the CI test slices (3484 +1827 + 168 doc), the
jq/http_clientslice for therg/json/sedchanges,
proptest_security,security_failpoint_tests, 95/95 grep spectests,
check-bash-compat("All non-excluded tests match real bash"),check-okf,check-doc-links.ssh_supabase_testscannot run here — noport-22 egress in this sandbox, the same condition CI skips the live SSH test
under.
Still open after this
The shared helper labels the resolved path, not the operand: in
/work,wc nopereportswc: /work/nope:where realwcechoeswc: nope:.Fixing it means threading the operand through 33 call sites, so it is left
out of this change.
cat,head,tailandgrepalready label correctly.Generated by Claude Code