Skip to content

chore(cats): classify nine verified false-positive families from run 20260730T220551Z - #640

Merged
ericfitz merged 1 commit into
mainfrom
chore/cats-fp-rules-20260730
Jul 31, 2026
Merged

chore(cats): classify nine verified false-positive families from run 20260730T220551Z#640
ericfitz merged 1 commit into
mainfrom
chore/cats-fp-rules-20260730

Conversation

@ericfitz

Copy link
Copy Markdown
Owner

Adds false-positive rules 61-69 from the 2026-07-30 CATS campaign
(20260730T220551Z, v1.6.2 @ 2de6d9c). True positives drop 670 -> 542.

Every rule was verified against the run's actual request and response bodies before
being written, and the whole set was dry-run against a throwaway copy of the database
first — the committed rules file was untouched until it passed:

  • 128 matches, matching the live reclassification exactly
  • no rule matched a 5xx (exit 0, no violations)
  • no rule exceeds the 5% share-of-true-positives threshold — the largest,
    FEEDBACK_TARGET_FIELD_CROSS_FIELD_400, is 31/670 = 4.6%
rule n basis
XXE_NOTES_CONTENT_NOT_PARSED 20 TMI has no XML parser; all 20 returned the payload HTML-escaped (]>&test;), zero entity expansion
FEEDBACK_TARGET_FIELD_CROSS_FIELD_400 31 target_field is valid only with target_type=threat_classification — a cross-field rule not expressible in OpenAPI
WHITESPACE_FIELD_NAMES_IGNORED_2XX 29 mangled key is an unknown field the decoder ignores; no additionalProperties:false in the contract
THREAT_MODEL_RESTORE_NOT_DELETED_409 14 restoring a non-deleted threat model is correctly a conflict
PREFERENCES_ALREADY_EXIST_409 11 /me/preferences is a per-user singleton: POST creates, PUT updates
MASS_ASSIGNMENT_DECLARED_WRITABLE_FIELD 10 timmy_enabled is a declared, non-readOnly property; CATS flags it on a name heuristic
IDOR_EMPTY_RESULT_SET 9 every finding returned total: 0 — nothing disclosed
MASS_ASSIGNMENT_SSE_STREAM_ENVELOPE 3 field-echo heuristic run over an SSE notAJson envelope
PREFERENCES_FREEFORM_JSON_ECHO 1 UserPreferences declares free-form per-client objects

The one rule worth a second look

WHITESPACE_FIELD_NAMES_IGNORED_2XX is the only rule that trades away a signal CATS
was right to look for. The question that decides it is whether a required field can
be mangled and still yield 2xx — that would be a validation bypass. It cannot today:
all 29 findings mangle an optional field only, zero mangle a required field. The
rule carries an inline note to re-verify that if the request models change.

Deliberately NOT suppressed

  • The 428 Not matching response schema findings — a CATS defect under research in
    chore: isolate CATS "Not matching response schema" false-positive trigger and report upstream #637. They are 79% of what remains, but suppressing them would hide genuine schema
    regressions in exactly the resources that matter most.
  • /saml/providers/{idp}/users 401s (19) and /me/groups/{id}/members 403s (12)
    — seeding and configuration gaps, not false positives. A rule here would mask real
    authorization bugs; the fixtures should be fixed instead.

Also noted

The older "not in a deleted state" rule is now dead — the server answers
"Threat model is not deleted". Rule 69 matches the current wording; the stale rule is
left in place rather than edited, since removing it would shift evaluation order.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WnrVyySuyJFqcMxB8q3pNc

…20260730T220551Z

Adds rules 61-69. Each was verified against the run's actual request and
response bodies before being written, and the set was dry-run against a
throwaway copy of the database first: 128 matches, no rule touching a 5xx,
and no single rule over the 5% share-of-true-positives threshold (the
largest, FEEDBACK_TARGET_FIELD_CROSS_FIELD_400, is 31/670 = 4.6%).

True positives drop 670 -> 542.

  XXE_NOTES_CONTENT_NOT_PARSED (20)
      TMI has no XML parser. All 20 responses returned the payload
      HTML-escaped ("]>&test;") with zero entity expansion.
  MASS_ASSIGNMENT_DECLARED_WRITABLE_FIELD (10)
      timmy_enabled is a declared, non-readOnly property of
      TeamProjectNoteBase and AssetBase. CATS flags it on a name heuristic.
  MASS_ASSIGNMENT_SSE_STREAM_ENVELOPE (3)
      chat/sessions streams text/event-stream; CATS runs a field-echo
      heuristic over the notAJson envelope, where no JSON object exists.
  IDOR_EMPTY_RESULT_SET (9)
      Every finding returned total 0 -- the filter matched nothing, so
      nothing was disclosed. Scoped to total==0 so a leak still surfaces.
  FEEDBACK_TARGET_FIELD_CROSS_FIELD_400 (31)
      target_field is valid only with target_type=threat_classification, a
      cross-field rule not expressible in OpenAPI.
  WHITESPACE_FIELD_NAMES_IGNORED_2XX (29)
      A whitespace-mangled field name is an unknown key the decoder ignores;
      TMI declares no additionalProperties:false. Verified all 29 mangled
      only OPTIONAL fields -- none bypassed a required field. The rule
      carries a note to re-verify that if the request models change.
  PREFERENCES_ALREADY_EXIST_409 (11)
      /me/preferences is a per-user singleton: POST creates, PUT updates.
  PREFERENCES_FREEFORM_JSON_ECHO (1)
      UserPreferences declares free-form per-client objects; size, client
      count and key pattern are still enforced in the handler.
  THREAT_MODEL_RESTORE_NOT_DELETED_409 (14)
      Restoring a non-deleted threat model is correctly a conflict. The
      older "not in a deleted state" rule no longer matches the server's
      current wording ("Threat model is not deleted").

Deliberately NOT suppressed:

  The 428 "Not matching response schema" findings -- a CATS defect being
  researched under #637. Suppressing them would hide genuine schema
  regressions in the resources that matter most.

  The /saml/providers/{idp}/users 401s (19) and /me/groups/{id}/members
  403s (12). These are seeding and configuration gaps, not false positives;
  a rule would mask real authorization bugs. Fix the fixtures instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WnrVyySuyJFqcMxB8q3pNc
@ericfitz
ericfitz merged commit f185479 into main Jul 31, 2026
12 checks passed
@ericfitz
ericfitz deleted the chore/cats-fp-rules-20260730 branch July 31, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant