Skip to content

feat(validator): add severity levels and suppressions - #30

Open
Statyk7 wants to merge 3 commits into
feat/feature-ownership-checksfrom
feat/validator-severity
Open

Statyk7 wants to merge 3 commits into
feat/feature-ownership-checksfrom
feat/validator-severity

Conversation

@Statyk7

@Statyk7 Statyk7 commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Closes #21. Stacked on #29: the base is feat/feature-ownership-checks, so this diff shows only the #21 changes. Merge #29 first; GitHub then retargets this PR to main.

Rule IDs and severities. Every finding has a stable rule ID, and a ruleSeverities map next to the rules table gives each one a severity: 14 errors and 3 warnings. Errors exit 2 and block the hook. Warnings are printed but exit 0. Output looks like this:

FFCA findings in features/nu/nu_presentation/pubspec.yaml:
  ⚠ [presentation-missing-own-domain] nu_presentation does not depend on nu_domain (...)
  → Add nu_domain under dependencies in the nu_presentation pubspec, or suppress this with the reason the screen does not use it.
FFCA: 0 errors, 2 warnings, 1 suppressed.

Suppressions. # ffca-ignore: <rule-id> — <reason> in a pubspec suppresses that rule for the package. The reason is required:

Case Result
No reason suppression-missing-reason error, suppresses nothing
Unknown or unsuppressible rule ID suppression-unknown-rule error
Matches no finding unused-suppression warning, --all only, since incremental mode does not compute every finding

New warning. presentation-missing-own-domain, held back from #24: a presentation package that does not depend on its feature's existing _domain. Presentation-only features have no domain, so they are not affected.

Hook. Stderr on exit 0 never reaches Claude, so validate_layers.sh now returns warnings as additionalContext JSON without blocking. hooks/validate_layers_test.sh, added in #17, gains a warn outcome and two cases against warning_workspace: a warning reaches Claude as context, and a suppressed finding stays silent.

Fixtures. The new warning_workspace must exit 0: it holds a warning, an error suppressed with a reason, and an unused suppression. invalid_workspace gains omicron (no reason) and pi (unknown rule ID). beta_presentation now depends on beta_domain so it does not pick up an accidental warning.

Docs. The README gains a rules, severities, and suppressions section with the full table of rule IDs. CLAUDE.md covers the warning path and the hook test, AGENTS.md lists the new files, and the auditor agent maps errors to fail and warnings to warn and lists suppressions with their reasons.

Out of scope. Suppressions apply per package, not per dependency line. // ffca-ignore in Dart files waits for #25, since no check reads Dart source yet.

Review fixes. Merges #29's review fixes (cycle attribution and 4-space pubspecs); the two new tests assert this branch's rule IDs. The review's suppression findings concern Dart-file suppressions and are fixed in #31.

Verified locally: dart analyze, dart format, dart test (48 tests, 11 new), both hook test scripts (validate_layers_test.sh: 17 cases, 2 new), markdownlint, and cspell. main is merged in through #29, resolving the overlap with #17.

Type of Change

  • New feature (feat)
  • Bug fix (fix)
  • Code refactor (refactor)
  • Documentation (docs)
  • CI change (ci)
  • Chore (chore)

🤖 Generated with Claude Code

https://claude.ai/code/session_017Vxxv9fVFGupLXEnmx5bSq

Every finding now carries a stable rule ID and a severity from the new
ruleSeverities map. Errors exit 2 and block the hook; warnings are
printed but exit 0. Output marks errors with ✗ and warnings with ⚠ and
ends with an error, warning, and suppressed count.

A `# ffca-ignore: <rule-id> — <reason>` comment in a pubspec suppresses
that rule for the package. A suppression without a reason or with an
unknown rule ID is an error and suppresses nothing; one that matches no
finding is a warning in --all mode.

Add the presentation-missing-own-domain warning held back from #24.

The hook returns warnings to Claude as non-blocking additionalContext,
covered by the new hooks/validate_layers_test.sh in the Hook Tests job.

Closes #21

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Vxxv9fVFGupLXEnmx5bSq
@Statyk7
Statyk7 requested a review from a team as a code owner October 14, 2026 19:08
…rity

# Conflicts:
#	AGENTS.md
#	CLAUDE.md
#	README.md
#	hooks/validate_layers_test.sh
…rity

Brings the review fixes from #29: deterministic cycle attribution and
4-space pubspec indentation. The cycle and indentation tests now assert
the rule IDs this branch adds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Vxxv9fVFGupLXEnmx5bSq
Statyk7 added a commit that referenced this pull request Oct 15, 2026
From a review of #29, #30 and #31:

- Check imports of undeclared workspace packages against the rules
  table. A Dart workspace resolves them, so `product_presentation` could
  import `cart_data` without either mode noticing. A forbidden layer
  reports that layer rule; an allowed one reports the new
  undeclared-dependency error.
- On a pubspec edit, run import checks and suppression validation for
  the edited package only, so a clean edit never blocks on a
  dependent's existing Dart findings.
- Skip missing-primary-barrel in incremental mode while a package has no
  lib/, so a new package can be scaffolded pubspec first.
- Read directives with a scanner that stops at the first declaration,
  follows multi-line directives, and reads every URI, so imports inside
  strings are ignored and both sides of a conditional import are checked.
- Check every barrel at the root of a data package's lib/, following
  relative exports one level, for exported DTOs and mappers.
- Report a `//` suppression naming a pubspec-only rule, and group
  suppression findings under the file the comment is in.
- Fix CLAUDE.md, which said dart only runs for pubspec edits, and cite
  the barrel rule's actual sources.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017Vxxv9fVFGupLXEnmx5bSq
Comment thread hooks/validate_layers.sh
# Exit 0 with output means warnings only. Stderr on exit 0 never reaches
# Claude, so hand the warnings over as additional context, without blocking.
if [[ -n "$output" ]]; then
jq -n --arg ctx "FFCA validator warnings (non-blocking):"$'\n'"$output" \

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before this change, output from a run that exited 0 was dropped. Now anything dart run prints (first-run/analytics notices, implicit Resolving dependencies... because scripts/ has a pubspec, SDK warnings) gets sent to Claude as "FFCA validator warnings". Can we forward only real findings? For example, only emit the JSON when $output contains FFCA findings, as the test's run_hook already checks. Or have the validator write findings to a dedicated stream so the hook doesn't depend on 2>&1.

if (violations.isEmpty) return 0; // silent pass
_report(violations, root);
return 2;
final result = _applySuppressions(violations, scope, reportUnused: false);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since warnings now go to Claude on every pubspec edit, could incremental mode only report warnings for the edited package (or only findings the edit could have caused)? Today, editing a shared package re-surfaces every existing warning across all its dependents each time, which seems likely to distract Claude into unrelated changes. Errors in dependents still make sense to report.


/// `# ffca-ignore: <rule-id> <separator> <reason>`, where the separator is an
/// em dash, an en dash, a hyphen, or a colon.
final _suppressionPattern = RegExp(r'#\s*ffca-ignore:\s*([A-Za-z0-9_-]+)(.*)$');

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small edge case: because the ID class includes -, a hyphen separator with no space before it (domain-uses-flutter- reason) is swallowed into the rule ID and reported as an unknown rule. Requiring the ID to end in [a-z0-9] (e.g. ([a-z0-9]+(?:-[a-z0-9]+)*)) would make the hyphen separator work as documented.

final String rule;
final String fix;

Severity get severity => ruleSeverities[id]!;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ruleSeverities[id]! turns a mistyped or unregistered rule id into a runtime crash, and the hook reads a crash as "skipping", so a missing map entry silently disables blocking for that rule. Could the IDs be an enum (with .id string and severity), or could we add a test that every id: passed to _Violation is a key of ruleSeverities?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants