Replace the scanner's A–F grade with status counts per requirement level

.specify/memory/constitution.md

@@ -32,7 +32,7 @@ own PR, and every product that renders the entity picks the field up.
 
 ### II. Every Claim Cites a Revision
 
-Every feature, scenario step, grade and matrix cell MUST link to the spec revision and section it
+Every feature, scenario step, scan result and matrix cell MUST link to the spec revision and section it
 is derived from. A claim without a citation does not ship. When a spec revision changes, the
 affected pages say so and keep rendering correctly against the revision they cite; nothing is
 silently re-interpreted against a newer revision. Dates on observations are mandatory.
@@ -128,4 +128,4 @@ Report in the PR description. MAJOR for a changed or removed principle, MINOR fo
 principle or section, PATCH for wording. Every plan's Constitution Check lists each principle and
 states how the feature complies or justifies the deviation in its Complexity Tracking table.
 
-**Version**: 1.0.0 | **Ratified**: 2026-10-13 | **Last Amended**: 2026-10-13
+**Version**: 1.0.1 | **Ratified**: 2026-10-13 | **Last Amended**: 2026-10-22

README.md

@@ -66,7 +66,7 @@ letsauth keeps one open dataset of what the agent protocols require and what rea
 | 🕸️ | **[Agent auth atlas](https://letsauth.eu/standards/)** | The specifications, drafts and EU acts behind agent authorization as one map of what builds on, profiles, competes with and replaces what, with maturity and dated revisions per document. Export: [`data.json`](https://letsauth.eu/standards/data.json). |
 | 🔗 | **[Delegation chain explorer](https://letsauth.eu/explorer/)** | Scenarios of how authority travels from a person to an agent to an API, step by step, with the decoded token at each hop and toggles for the attacks and misconfigurations that break the chain. Every step cites a specification revision. Export: [`data.json`](https://letsauth.eu/explorer/data.json). |
 | 🩻 | **[Breach anatomy](https://letsauth.eu/breaches/)** | Published vulnerabilities and incidents in how agents were authorized, each written from its advisory and traced to the control that was missing, the atlas requirements that stand for it, and a replay toggle in the explorer. Export: [`data.json`](https://letsauth.eu/breaches/data.json). |
-| 🔍 | **[Scanner](https://letsauth.eu/mcp/scanner/)** | A passive scanner that reads each listed server's public OAuth metadata every night, grades it against the MCP Authorization specification and opens one pull request of observations. The grade shows next to each server in the atlas; one report per server at `/mcp/servers/<id>/scan/` and `/mcp/servers/<id>/scan.json`. |
+| 🔍 | **[Scanner](https://letsauth.eu/mcp/scanner/)** | A passive scanner that reads each listed server's public OAuth metadata every night, checks it against the MCP Authorization specification and opens one pull request of observations. Nothing is graded: the atlas shows each status, and every report counts them per requirement level; one report per server at `/mcp/servers/<id>/scan/` and `/mcp/servers/<id>/scan.json`. |
 
 Next on the [roadmap](docs/roadmap.md): agents under EU rules, and a changelog generated from
 the data.
@@ -93,7 +93,7 @@ You can download everything the site shows, under CC BY 4.0:
 curl -s https://letsauth.eu/mcp/data.json | jq '.servers | length'
 curl -s https://letsauth.eu/standards/data.json | jq '.relations[0]'
 curl -s https://letsauth.eu/explorer/data.json | jq '.scenarios[0].toggles | length'
-curl -s https://letsauth.eu/mcp/servers/notion/scan.json | jq '.grade'
+curl -s https://letsauth.eu/mcp/servers/notion/scan.json | jq '.summary'
 ```
 
 Or read the YAML directly: one file per entity under [`data/`](data/), with the schema in

docs/README.md

@@ -5,7 +5,7 @@
 | [`architecture.md`](architecture.md) | You want the stack, the repository layout and how data flows from a pull request to a page. |
 | [`data-model.md`](data-model.md) | You touch anything under `data/` or `src/schema/`. The model every product is a view on. |
 | [`atlas-maintenance.md`](atlas-maintenance.md) | You add a server, a feature, a spec, a relation or an observation, or review a data pull request. |
-| [`scanner.md`](scanner.md) | You work on the passive scanner, its grade or the nightly job. |
+| [`scanner.md`](scanner.md) | You work on the passive scanner, its report summary or the nightly job. |
 | [`development.md`](development.md) | You set up a checkout, run the tests or refresh screenshot baselines. |
 | [`deploy.md`](deploy.md) | You host the site or rotate a secret. |
 | [`roadmap.md`](roadmap.md) | You want to know what comes next and why in that order. |

docs/architecture.md

@@ -57,7 +57,7 @@ that has no page of its own.
 /mcp/features/<id>/             one requirement
 /mcp/revisions/<id>/            the matrix as of an older revision (the current one redirects to /mcp/)
 /mcp/changes/<from>/<to>/       what changed between two revisions
-/mcp/scanner/                   what the scanner reads, how it grades, and every report
+/mcp/scanner/                   what the scanner reads, how it reports, and every report
 /mcp/data.json                  the atlas export
 /standards/                     Agent auth atlas: the standards map
 /standards/<id>/                one specification
@@ -96,7 +96,7 @@ src/
                  page head and foot every page uses are in styles/global.css
   lib/           pure functions over the dataset: atlas matrix, standards graph, chains, exports, citations
   pages/         routes; file-system routing, every page pre-rendered
-  scanner/       the passive scanner: url guard, fetch, checks, grade, observations, nightly job
+  scanner/       the passive scanner: url guard, fetch, checks, summary, observations, nightly job
   schema/        Zod schemas, the contract for data/
   styles/        tokens.css (the one token system), global.css (shared surfaces), fonts.css
 tests/
@@ -133,7 +133,7 @@ the `WWW-Authenticate` challenge), never a token, a body or an authorization flo
 passively observable requirement of the current MCP Authorization revision. It runs in exactly
 one place: a Scaleway Serverless Job in an EU region, every night, from the image `Dockerfile`
 builds. A run writes observations and opens one pull request; nothing on the site triggers a
-scan. [`scanner.md`](scanner.md) has the request list, the grade and the job.
+scan. [`scanner.md`](scanner.md) has the request list, the summary and the job.
 
 ## Hosting and delivery
 

docs/atlas-maintenance.md

@@ -108,7 +108,7 @@ builds on, profiles or replaces, in the row of its publishing body.
   `observedAt` and the page that shows the change; git keeps the history. Leave the field out
   rather than guess: a missing class renders as a dashed "unknown" and the server is still
   scanned. `auth: api-key` or `auth: none` takes the server out of the nightly scan and gives it
-  the "No OAuth" grade; when a scan's run summary says the endpoint's answer suggests another
+  "No OAuth" in its scan summary; when a scan's run summary says the endpoint's answer suggests another
   class, check the page and correct the file, not the observations.
 - `transport: stdio-only` makes every cell `not-applicable` without observations.
 - The seed list came from `jaw9c/awesome-remote-mcp-servers` (research R4). To re-seed from a
@@ -187,4 +187,4 @@ further commit (or merge the base branch) to get a green run.
 
 Scanner pull requests must not need new baselines. The scanner screenshots set every
 `[data-nightly]` element (counts and dates the nightly run changes) to a fixed value and mask the
-grades and report rows, so mark any new nightly-changing text on those pages with `data-nightly`.
+counts and report rows, so mark any new nightly-changing text on those pages with `data-nightly`.

docs/data-model.md

@@ -104,7 +104,7 @@ observation is one about a feature: `{ class, method, evidence, observedAt, obse
 page and corrected in place by pull request; git keeps the history. A server without one is
 rendered as a *derived* `unknown`, like a cell without an observation, never as a verdict. The
 atlas filters and groups by both classes; the scanner scans a server unless its `auth` is
-`api-key` or `none`, grades none of those ("No OAuth"), and reports in its run summary when what
+`api-key` or `none`, scans none of those ("No OAuth"), and reports in its run summary when what
 an endpoint answered does not sit with the stated class. The scanner never writes a
 classification.
 

docs/design.md

@@ -258,7 +258,7 @@ package of the redesign that builds it. Anything not in this table does not anim
 | M9 | Matrix | Row and column crosshair on hover/focus of a cell (`--signal-soft` background, 120ms) | orientation in a large grid | instant | WP4 |
 | M10 | Matrix | Cell dialog: backdrop fades, panel moves up 8px, 220ms, using `@starting-style` | "a detail opened" | appears | WP4 |
 | M11 | Matrix filters | Count text changes with no animation; hidden rows collapse instantly (today's behaviour) | n/a | n/a | WP4 |
-| M12 | Scanner report | The request ladder prints line by line, 60ms apart, when the section scrolls into view, once; the grade appears last | "this is what the scanner did, in order" | all lines shown | WP7 |
+| M12 | Scanner report | The request ladder prints line by line, 60ms apart, when the section scrolls into view, once; the count of checks appears last | "this is what the scanner did, in order" | all lines shown | WP7 |
 | M13 | Breach page | Timeline rail draws in top to bottom, 480ms, dates fade in behind it | "events in sequence" | shown | WP6 |
 | M14 | Long-form pages (`Document.astro`) | Reading progress bar, 2px `--signal`, `animation-timeline: scroll(root)` behind `@supports` | place in the text | hidden | WP2 |
 | M15 | Controls | Colour and border transitions on hover/focus, 120ms | affordance | instant | WP2 |

docs/development.md

@@ -43,7 +43,7 @@ set.
 
 - **Unit tests** live in `tests/unit/` with fixtures in `tests/fixtures/`. They cover the matrix
   derivation, the standards graph, exports, citations, data validation and every scanner module
-  (guard, urls, checks, grade, append rule, nightly job). The scanner tests never make a network
+  (guard, urls, checks, summary, append rule, nightly job). The scanner tests never make a network
   request: fetch is stubbed with fixture responses.
 - **End-to-end tests** live in `tests/e2e/` and run against the built site. They check that pages
   read without JavaScript, that the islands work with it, accessibility with axe, and
@@ -80,7 +80,7 @@ that workflow does not start CI, so push any further commit (or merge the base b
 green run. Update baselines only for intended changes.
 
 Scanner pull requests must not need new baselines: the scanner pages mark text the nightly run
-changes with `data-nightly`, and the screenshot tests fix those values and mask the grades.
+changes with `data-nightly`, and the screenshot tests fix those values and mask the report links.
 
 ## Working on the scanner
 

docs/scanner.md

@@ -126,7 +126,7 @@ A Scaleway Serverless Job runs `pnpm scan:nightly` (`scripts/scan-nightly.ts`) e
 4. when nothing under `data/observations/` changed, logs "nothing to append" and stops;
 5. otherwise commits to a new branch `scanner/<date>-<hhmm>` (UTC), pushes it, and opens one pull
    request `Scanner: <date> <hh:mm> UTC` whose description is the run summary (per server the
-   grade, the cells appended and the checks that could not be judged);
+   scan summary, the cells appended and the checks that could not be judged);
 6. keeps earlier scanner pull requests and branches open: each run compares against `main`,
    and later results do not include every earlier unmerged observation. Review and merge pending
    evidence in chronological order, resolving append-only list conflicts by retaining both runs.
@@ -218,27 +218,31 @@ Set up once, by the owner:
 The workflow never creates or changes the job's schedule, resources or secret; it only moves
 the image forward, so those settings stay where the owner put them.
 
-## The grade
-
-Shown on report pages, on atlas server pages and next to each server name in the atlas matrix,
-never used to sort or rank:
-
-1. F when a MUST-level check is unsupported or partial.
-2. Withheld (no letter) when a MUST-level check is unknown.
-3. C when every MUST-level check is supported and a SHOULD-level check is unsupported or partial.
-4. B when every MUST-level check is supported, no SHOULD-level check fails, and a SHOULD-level
-   check is unknown.
-5. A when every MUST-level and SHOULD-level check is supported.
-
-A MUST the server may meet in either of two ways is met when either way is supported, and counts
-as unknown while the other way is unknown. Protected resource metadata may be served at a
-well-known location or announced in the `WWW-Authenticate` challenge; the scanner reads both, so
-a server that offers only the challenge is graded, one where the challenge could not be judged
-and the well-known document is missing is withheld, and only a server that offers neither fails
-on discovery. MAY-level and deprecated requirements are listed and never change
-the grade. Servers whose `auth` class (`docs/data-model.md` › Server, roadmap 009) is `api-key`
-or `none` are neither scanned nor graded ("No OAuth"); the class is an evidenced statement on the
-server file.
+## The summary
+
+The scanner does not grade, score or rank. A report shows each check with its status, and a
+summary that counts those statuses per requirement level (spec 017, which replaced the A–F grade
+of 004 FR-008):
+
+```
+MUST    8 checks   supported 7, unsupported 1
+SHOULD  4 checks   unsupported 3, unknown 1
+MAY     2 checks   unsupported 2
+```
+
+Under the counts, each level names the checks that are not supported, linked to their rows. The
+levels are never added up into one number, and nothing on the site sorts servers by a scan. The
+atlas matrix shows the cells themselves and a quiet "report" link next to each server name; the
+server page shows how many checks ran and when.
+
+`unknown` is counted as unknown, never as unsupported. A MUST the server may meet in either of
+two ways counts as supported when either way is supported, and as unknown while the other way is
+unknown. Protected resource metadata may be served at a well-known location or announced in the
+`WWW-Authenticate` challenge; the scanner reads both, so a server that offers only the challenge
+counts as meeting discovery, and the summary says in a sentence when a count differs from its
+row. Deprecated requirements are counted under their own level. Servers whose `auth` class
+(`docs/data-model.md` › Server, roadmap 009) is `api-key` or `none` are not scanned and read
+"No OAuth"; the class is an evidenced statement on the server file.
 
 ## What there is not
 

scripts/scan.ts

@@ -76,8 +76,9 @@ function printReport(report: Report): void {
     `${report.scanner} · ${report.target.url}${report.target.server ? ` (${report.target.server})` : ""}`,
   );
   out.push(`against ${report.revision.spec} ${report.revision.revision} · ${report.generatedAt}`);
-  out.push(`grade: ${report.grade.label}`);
-  for (const r of report.grade.reasons) out.push(`  ${r}`);
+  out.push(`summary: ${report.summary.label}`);
+  for (const c of report.summary.open) out.push(`  ${c.level} ${c.status}: ${c.name}`);
+  for (const n of report.summary.notes) out.push(`  ${n}`);
   out.push("");
   for (const c of report.checks) {
     out.push(`${STATUS_WORD[c.status] ?? c.status}  ${c.level.padEnd(6)} ${c.feature}`);

specs/004-scanner/contracts/report.md

@@ -2,12 +2,12 @@
 
 One shape for the command's output and the per-server JSON on
 the site. Produced by `scan()` in `src/scanner/` and, for recorded reports, by `latestScan()` in
-`src/lib/scanner.ts` (which fills `checks[]` and `grade` from observations and leaves
+`src/lib/scanner.ts` (which fills `checks[]` and `summary` from observations and leaves
 `requests` and `discovered` empty, since the site never contacts a server).
 
 ```jsonc
 {
-  "version": 1,
+  "version": 2,                           // 2 since spec 017: `summary` replaced `grade`
   "scanner": "letsauth-scanner/0.1.0",
   "generatedAt": "2026-10-15T03:17:04Z",
   "recorded": false,                       // true when built from the repository's observations
@@ -58,10 +58,17 @@ the site. Produced by `scan()` in `src/scanner/` and, for recorded reports, by `
       "evidence": "https://mcp.example.com/.well-known/oauth-protected-resource/mcp — HTTP 200; found at the path-aware location",
       "observedAt": "2026-10-15T03:17:04Z" }
   ],
-  "grade": {
-    "letter": "C",                         // "A" | "B" | "C" | "F" | null
-    "label": "C",                          // or "Withheld" / "Not scanned" / "No OAuth"
-    "reasons": ["SHOULD-level check unsupported: Client ID Metadata Documents supported"]
+  "summary": {                             // spec 017: counts only, no grade or score
+    "state": "scanned",                    // or "not-scanned" / "no-oauth"
+    "label": "MUST 8 supported · SHOULD 3 supported, 1 unsupported",
+    "tallies": [                           // one per level present, MUST first
+      { "level": "MUST", "total": 8, "counts": [{ "status": "supported", "n": 8 }] }
+    ],
+    "open": [                              // checks not supported, in report order
+      { "feature": "mcp.as.cimd", "name": "Client ID Metadata Documents supported",
+        "level": "SHOULD", "status": "unsupported" }
+    ],
+    "notes": []                            // the either-or rule in words, or why there are no counts
   },
   "observations": [                        // the same checks as data-model observations;
     { "server": "example", "feature": "mcp.prm.well-known", "revision": { … },

specs/004-scanner/spec.md

@@ -206,15 +206,17 @@ append-only check passes.
   evidence link and the date of the newest scanner observation, and the report MUST show a grade
   with its reasons or say why the grade is withheld; a server without scanner observations MUST
   say so and show no grade.
-- **FR-008**: The grade MUST be derived only from the statuses the report shows, by this rule, and
+- **FR-008** *(superseded by 017-neutral-scan-summary, 2026-10-22: there is no grade; the report
+  counts statuses per level)*: The grade MUST be derived only from the statuses the report shows, by this rule, and
   the rule MUST be published next to every grade: a MUST-level check that is unsupported or partial
   gives F; otherwise a MUST-level check that is unknown withholds the grade; otherwise a
   SHOULD-level check that is unsupported or partial gives C; otherwise a SHOULD-level check that is
   unknown gives B; otherwise A. MAY-level and deprecated requirements never affect the grade. A
   MUST the server may meet in either of two ways (protected resource metadata at a well-known
   location or in a `WWW-Authenticate` challenge) counts as unknown while the other way is unknown.
   A server listed as open or API-key is not graded ("No OAuth").
-- **FR-009**: The atlas matrix MUST show each listed server's latest grade (or that it is not
+- **FR-009** *(the grade part superseded by 017-neutral-scan-summary: the matrix links the report
+  and shows no grade)*: The atlas matrix MUST show each listed server's latest grade (or that it is not
   scanned or not graded) next to its name, linked to its report, and the site MUST NOT order or
   rank servers by grade anywhere. The scanner page MUST NOT carry a second list of servers; it
   describes the method and points at the atlas.