Skip to content

test(grep): compare _grep_python against a real grep (closes #63) - #79

Merged
shauryagangrade merged 2 commits into
shauryagangrade:mainfrom
dchaudhari7177:test/grep-differential
Aug 14, 2026
Merged

test(grep): compare _grep_python against a real grep (closes #63)#79
shauryagangrade merged 2 commits into
shauryagangrade:mainfrom
dchaudhari7177:test/grep-differential

Conversation

@dchaudhari7177

Copy link
Copy Markdown
Contributor

Closes #63.

What

Eleven differential cases over one fixture corpus — text, nested dirs, a NUL-bearing binary, files in and out of a glob — requiring _grep_python and the system grep to agree on: plain regex, -i, glob filter, no-match, binary skipping, regex metacharacters (a[0-9]b, ^needle, line$, n..dle), and a single-file target.

Output from both sides is reduced to {(relative posix path, lineno, text)} before comparing. grep joins its argument to each entry with / while os.walk uses os.sep, so the raw strings differ on Windows even when the matches are identical — normalising keeps the assertion about which lines matched, which is the thing that can actually drift.

Two tests assert the concrete file set as well as agreement, so "both implementations agree on nothing" can't pass.

One deliberate departure from the issue

The acceptance criterion says "passes on macOS/Linux, skips cleanly on Windows". I gated on grep being present instead of on sys.platform, for two reasons:

  • Git for Windows ships GNU grep, so a platform gate skips exactly where _grep_python is the default code path and divergence matters most.
  • Gating by platform would have meant I couldn't run these at all — I'm on Windows. As written they run green here (11 passed), and they skip cleanly anywhere grep genuinely is absent, which is the same protection.

Say the word if you'd rather have the platform gate and I'll swap it.

It already found something

Given a single file rather than a directory, GNU grep drops the filename prefix:

grep:          1:needle at the top
_grep_python:  C:\...\top.txt:1:needle at the top

So grep(pattern, path="a.txt") returns a different shape depending on whether a grep binary happens to be installed — precisely the class of drift this issue exists to catch.

I pinned it in test_single_file_output_format_diverges rather than fixing it: aligning the two changes the grep tool's output, which is your call rather than a drive-by in a test-only PR. Adding -H to the tool's flags would do it, and I'm happy to send that separately.

Verification

pytest tests/test_grep_differential.py11 passed. ruff check clean.

Full suite on this branch: 63 passed, 2 failed — the 2 are test_tools.py::test_grep / ::test_grep_ignore_case, which fail on a Windows checkout of main for an unrelated reason I've sent as #78. On a Linux runner the whole suite should be green.

dchaudhari7177 and others added 2 commits August 14, 2026 22:45
…angrade#63)

_grep_python is the fallback for machines with no grep binary, so the two
implementations serve the same calls and were only ever checked against
hand-written expected strings. That lets them drift on exactly the axes the
issue names: regex flavour, glob handling, case folding, binary detection.

Eleven cases over one corpus (text, nested dirs, a NUL-bearing binary, files
in and out of a glob). Output is reduced to {(relative posix path, lineno,
text)} before comparing, since grep joins with "/" and os.walk with os.sep --
the raw strings differ on Windows even when the matches are identical, and
the assertion is about which lines matched.

Gated on grep being *present* rather than on sys.platform, which is a
deliberate departure from the issue text: Git for Windows ships GNU grep, so
skipping by platform would skip exactly where the fallback is the default
path and divergence matters most. These run green here on Windows.

Two anchors keep "both agree on nothing" from passing: the plain-regex and
glob cases assert the concrete file set as well as agreement.

The comparison already found one real divergence. Given a single file rather
than a directory, GNU grep drops the filename prefix ("1:needle at the top")
while _grep_python always emits it -- so the grep tool's output shape depends
on whether a grep binary is installed. Pinned in its own test rather than
fixed: aligning them changes tool output and is the maintainer's call.
@shauryagangrade
shauryagangrade merged commit 76335db into shauryagangrade:main Aug 14, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Testing]: Differential tests — _grep_python vs system grep

2 participants