Skip to content

fix(test): reject empty --filter before any requests - #660

Open
WUWTest wants to merge 4 commits into
TestSprite:mainfrom
WUWTest:fix/empty-name-filter
Open

WUWTest wants to merge 4 commits into
TestSprite:mainfrom
WUWTest:fix/empty-name-filter

Conversation

@WUWTest

@WUWTest WUWTest commented Oct 24, 2026 •

Copy link
Copy Markdown

What does this PR do?

An empty shell variable in --filter "$FILTER" currently runs every project test, or is silently ignored on a single-test invocation. Refuse an explicitly empty value in the test run, test rerun, and ci run actions before any request, including with --dry-run.

Use the existing VALIDATION_ERROR (exit 5, details.field: filter) and explain how to omit the flag or supply a non-empty substring. Omitted and non-empty values, including whitespace-only substrings, retain their behavior. Update all three help texts, their snapshots, and the command reference.

Related issue

Closes #606

Season 4 CLI Improvement Bonus

  • I'm entering this PR for the Season 4 CLI Improvement Bonus

Discord username:

Type of change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that changes existing behavior)
  • Documentation only
  • Build / CI / chore

Checklist

  • PR targets the main branch.
  • Commits follow Conventional Commits.
  • npm run lint and npm run format:check pass.
  • npm run typecheck passes.
  • npm test passes and coverage stays at or above the 80% gate.
  • New behavior is covered by unit tests (mock-based; no network or credentials required).
  • No secrets, API keys, internal endpoints, or personal data are included.
  • User-facing changes are reflected in DOCUMENTATION.md.

Notes for reviewers

The runtime change follows the accepted scope: one helper in test.ts, used by the three command actions; no selector refactor or new dependencies. The issue is assigned to WUWTest. Thanks to OkeyAmy for the report and reproducer.

Validation on macOS, Node v25.9.0:

  • Before the runtime fix, all 15 new empty-value regression cases fail on a18d37c; the 9 selection controls pass.
  • After the fix, the 24 new cases pass. They cover project/single-test paths, dry-run, --filter=, and omitted/non-empty/whitespace-only controls with injected HTTP fixtures.
  • Full unit suite with coverage: 4,822 passed, 2 skipped. Statements 90.09%, branches 85.10%, functions 83.65%, lines 92.27%.
  • Local E2E suite (including build): 110 passed, 1 skipped.
  • Typecheck, lint, format check, and git diff --check pass. Lint reports the existing unused-disable warnings (0 errors). Exactly three help snapshots changed.

Summary by CodeRabbit

  • Bug Fixes
    • test run, test rerun, and ci run now reject an explicitly empty --filter with a validation error before making a request, including in dry-run and single-test cases.
    • Non-empty filters match case-insensitive substrings, and whitespace is preserved. When the filter is omitted, commands continue to run all tests.
  • Documentation
    • Updated command help and documentation to clarify filter behavior, including that filters must be non-empty when specified.

Document empty --filter validation and unchanged omitted/non-empty behavior for test run, test rerun, and ci run.
Validate --filter in the test run, test rerun, and ci run actions with the existing exit-5 validation error. Preserve omitted and non-empty values, including whitespace, and cover the command paths with offline regression tests.

Refs TestSprite#606
Update only test run, test rerun, and ci run help snapshots to match the clarified --filter option.
@github-actions

Copy link
Copy Markdown

🏆 Season 4 CLI Improvement Bonus: entry recorded. Thanks, @WUWTest!

To be counted, this PR needs the following by Oct 29, 11:59 PM PT:

  • ✅ A linked issue assigned to you.
  • ✅ Out of draft.
  • ℹ️ No Discord username in the description. Add one so we can reach you, or we'll reach you here on the PR.

Rewards go to merged PRs judged a genuine improvement, and up to three bonus-eligible PRs count per person. Untick the box to withdraw the entry. This comment refreshes when you edit the PR. Full rules.

@github-actions github-actions Bot added the s4-cli-bonus Entered for the Season 4 CLI Improvement Bonus label Oct 24, 2026
@coderabbitai

coderabbitai Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 06fb0cb and 66e662e.

Files selected for processing: 1
  • src/commands/ci.ts
Files skipped from review as they are similar to previous changes: 1
  • src/commands/ci.ts

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Walkthrough

The test run, test rerun, and CI commands now reject explicitly empty --filter values before making a request. Omitted filters, non-empty filters, and whitespace-only values retain their existing meanings.

Changes

Name Filter Validation

Layer / File(s) Summary
Validate test and CI command filters
src/commands/test.ts, src/commands/ci.ts
parseNameFilter rejects an empty value. The test run, test rerun, and CI handlers pass the parsed filter to their existing execution paths. Help text describes the non-empty filter rule.
Verify and document filter behavior
src/commands/test.filter.spec.ts, DOCUMENTATION.md
Tests cover rejection before requests, omitted filters, case-insensitive matching, and preserved whitespace. Documentation describes the filter rules for test run, test rerun, and ci run.

Priority: Normal

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related issues

  • TestSprite/testsprite-cli#657 — This PR changes empty-filter handling in the shared selection interface. The preview-selection feature described by the issue remains unimplemented.
Merge Risk: Minimal · up to 66e66

Empty filters are rejected before requests in the inspected commands. No issue identified here prevents merging after normal checks.

Pre-merge checks: Passed 5
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: rejecting explicitly empty --filter values before requests. It matches the changes across test run, test rerun, and ci run.
Linked Issues check Passed Issue #606 requires rejection of only an explicitly empty --filter for test run, test rerun, and ci run. The change adds parseNameFilter, uses it in the three command handlers before existin…
Out of Scope Changes check Passed The changes stay within issue #606. They add shared validation, focused CLI regression tests, and related help and documentation updates for the three specified commands. The summary reports no select…
Docstring Coverage Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 3 functions across 2 files.
Full details: Linked Issues check

Explanation

Issue #606 requires rejection of only an explicitly empty --filter for test run, test rerun, and ci run. The change adds parseNameFilter, uses it in the three command handlers before existing filter checks, and returns the existing validation error. The summary confirms exit 5, details.field: filter, no request before failure, and coverage of single-test, --all, normal, and dry-run paths. Omitted, non-empty, and whitespace-only values remain supported. The change also updates the three help texts, snapshots, and DOCUMENTATION.md.

Full details: Out of Scope Changes check

Explanation

The changes stay within issue #606. They add shared validation, focused CLI regression tests, and related help and documentation updates for the three specified commands. The summary reports no selector refactor, new dependency, read-only list command change, or --status change. The added comment and test fixtures support the scoped implementation and verification.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Add the missing docstring for the command factory touched by the filter fix, following the review coverage advisory. No runtime behavior changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

s4-cli-bonus Entered for the Season 4 CLI Improvement Bonus

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Hackathon] fix(test): --filter "" silently widens test run --all, test rerun --all, and ci run to every test

1 participant