chore(cats): track upstream Endava/cats#206 — CATS validates 2xx responses against `example`, not `schema`

[ericfitz]

Summary

Tracking issue for the upstream CATS defect root-caused in #637 and filed as
Endava/cats#206.

CATS validates a 2xx response against the operation's declared example, not its schema.
When an example is present, the response body's property names (recursively) must be a subset of
the example's. The schema is never consulted — replacing it with {"type":"object"} or even true
still fails. When no example is present the check never runs at all.

This produced 428 of the 542 true positives (79%) in the 20260730T220551Z baseline, all
reported as "Not matching response schema" — a misleading label, since the schema was never the
thing being checked.

Why this is still open after #654

#654 remediated the findings legitimately rather than by suppression: the 23 genuinely
incomplete response examples in api-schema/tmi-openapi.json were completed, so the findings clear
and real schema regressions still surface. No false-positive rule was added.

That leaves two things to come back to:

1. One site is unsatisfiable from the spec side. MinimalDiagramModel.metadata is
   additionalProperties: {type: string} — a free-form map whose keys are unbounded, so no example
   can enumerate them. Expected to be the residual ~4 findings on
   GET /threat_models/{id}/diagrams/{id}/model. These cannot be fixed until upstream does.
2. A standing maintenance burden. Every response example must now stay exhaustive relative to
   its schema, or CATS re-reports the operation. Nothing enforces that today, so the next added
   optional property silently reintroduces the finding.

Acceptance Criteria

• Monitor Endava/cats#206 for an upstream fix
• Consider contributing the fix upstream — the reproduction is already minimal (1 path, 1
  schema, a 20-line Python stub; deleting the example alone flips fail → pass)
• Once fixed, re-evaluate whether the completed examples should stay. They are good docs
  independent of CATS, so the likely answer is yes — but the obligation to keep them exhaustive
  goes away, and the MinimalDiagramModel.metadata residual should clear
• Decide whether to add a spec lint asserting response examples remain exhaustive, if we stay on
  an unfixed CATS for long

Related upstream defects

Two further CATS defects were noted in the same upstream issue as possibly sharing a code path, and
should be tracked alongside it:

• ExamplesFields posts a schema-level object example as the whole body where the request
  schema is an array — produces 400 "value must be an array" on assets/bulk and
  documents/bulk.
• Array query params are comma-joined regardless of explode: true — sends
  ?severity=high%2Chigh instead of severity=high&severity=high.

References

• chore: isolate CATS "Not matching response schema" false-positive trigger and report upstream #637 — the finding cluster this root-caused
• fix(auth): require auth on /saml/providers/{idp}/users and complete response examples #654 — the remediation (23 examples completed), merged as 39d2d93 (v1.6.4)
• HANDOFF.md — full triage, including the Note-vs-TeamNote discriminator this explains