Skip to content

ci: add a deterministic docs linter, replacing the redirect checker - #211

Merged
ManuelGerding merged 6 commits into
mainfrom
ci/docs-lint
Sep 3, 2026
Merged

ManuelGerding merged 6 commits into
mainfrom
ci/docs-lint

Conversation

@ManuelGerding

Copy link
Copy Markdown
Member

docs-public had no CI on pull requests at all — the only workflow was workflow_dispatch only and probed the published site. Every one of the ~45 correctness defects found in the recent read-through (#208, #209, #210) could have been caught before merge, and nothing was looking.

This adds a deterministic linter, replaces the flaky redirect checker, and writes the conventions down.

Only what the change introduces

Every check runs against two trees and the results are diffed, so a pull request is judged on what it adds. Two consequences:

  • Existing debt never fails a build. There are 159 pre-existing findings today (78 heading case, 35 table width, 33 product-name, 11 whitespace, 1 table-columns, 1 spelling). style: consistent product names, arrows, dashes and whitespace #210 clears 34 of those; this PR clears 38 more. Nobody has to fix the rest to turn the gate on.
  • A stricter convention can be adopted without a repo-wide cleanup first — which is what makes the heading-case rule practical at all.

Finding identity deliberately excludes the line number, so moving a paragraph doesn't resurface everything below it as new.

The baseline is the target branch tip, not the merge base

This is the one design decision worth pausing on. actions/checkout gives us the PR merged into main, and we compare that against main itself — not against the merge base.

That is the only way to catch what happened between #207 and #208: #207 changed a heading's level on a line #208 had just corrected, from a base that predated it. Git reported no conflict, the typo came back, and nobody noticed until a checker found it two days later. Against the merge base that regression is invisible; against main it is a new finding. Test T2 below is exactly that scenario.

Checks

Check Scope
link, anchor relative links and #anchors resolve, under both slug conventions in use here
json ```json samples parse (deliberate ... elisions skipped)
spelling denylist of misspellings we have actually had — everywhere, code fences included
us-english British forms — prose only
product-name WebSocket, Docker Compose, Helm, GitHub, OpenAPI — prose only, skipping code spans, URLs and identifiers
heading-case title case, exempting any heading containing a ?
table-width, table-columns rows share a width and a column count
trailing-whitespace outside fences, keeping real Markdown hard breaks
redirect-target every .gitbook.yml target resolves to a file
moved-file a removed or renamed page has a redirect

Why spelling and us-english differ in scope: labelled: true could be a real config key, and renaming someone's field is not a docs decision — whereas Kuberneters is never a valid identifier. That split also closes a blind spot: the Kuberneters typos #208 fixed lived in // comments inside query-language code fences, which a prose-only scan would miss.

Stdlib only, no dependencies, ~2s for the whole repo. Runnable locally:

python3 .github/scripts/docs_lint.py --base origin/main

Replacing the redirect checker

redirect-url-checker.yml is removed. It probed the published site over the network, which GitBook rate-limits, so it never passed reliably — and being workflow_dispatch only, in practice it never ran on a pull request. Its purpose is now covered locally and deterministically by redirect-target and moved-file, which read .gitbook.yml and the tree instead of making requests.

38 live 404s, found by the new check

redirect-target immediately found 38 redirects aimed at files that no longer exist — those old URLs 404 today. All fallout from the reorganisation that introduced concepts/ and quick-start/, across only six distinct targets:

use-steadybit/actions.md                      -> concepts/actions/README.md                    (25)
use-steadybit/discovery/README.md             -> concepts/discovery/README.md                   (6)
getting-started.md                            -> quick-start/getting-started.md                 (3)
troubleshooting/agent.md                      -> troubleshooting/common-fixes/agents.md         (2)
use-steadybit/experiments/file-import-export  -> .../experiments/share/file-import-export       (1)
integrate-with-steadybit/webhooks.md          -> integrate-with-steadybit/webhooks/README.md    (1)

Every destination verified to exist. No redirect key changes, so no currently-working URL is affected. It's a separate commit if you'd rather handle it apart from the CI change.

Conventions

CLAUDE.md gains a Writing Conventions section, so the linter, contributors and any future review agent share one source. It records the three decisions that were open:

  • Headings — title case, except question-style headings, which stay sentence case. The troubleshooting pages are written as questions and title-casing them reads wrong.
  • US English — already true throughout, but never written down, so nothing stopped it drifting back.
  • Oxford comma — the convention, marked explicitly as not linted. I built the check, then deleted it: all six sampled hits were false positives (open- and closed-source extensions, begin and run your first experiments, and one that already had the comma). It is documented for humans and any later prose review instead.

Verification

Test Result
T1 clean branch vs main 0 ✓
T2 reintroduce a fix already on main (the #207 case) 2 — the typo and the table drift it caused ✓
T3 move a page, no redirect 3 — missing redirect + 2 dangling links ✓
T4 new sentence-case heading 1 ✓
T5 new question-style heading 0 ✓
T6 British spelling in prose 1 ✓
T7 typo inside a code fence 2 ✓
T8 denylisted string in an identifier, a URL, and a fenced YAML key 0 ✓
T9 redirect to a nonexistent file 1 ✓

No conflict with #210 — the two touch disjoint files and can merge in either order.

Two caveats I'd own

  • The denylist is a maintenance debt. Keep it short and high-signal; a stale entry erodes trust in the gate faster than a missing one costs.
  • heading-case is the least mechanical check here. If it proves noisy on real pull requests, demote it to a warning rather than letting people learn to ignore a red check.

After this and #210, the standing baseline is ~108 findings — almost entirely heading case (77) and table padding (30). None of it blocks anything.

Checks the things that were wrong across the docs and can be verified
without judgment:

- relative links and `#anchors` resolve
- ```json samples parse
- a denylist of misspellings we have actually had
- product-name casing in prose (WebSocket, Docker Compose, Helm, GitHub,
  OpenAPI), skipping code spans, URLs and identifiers
- headings are title case, except question-style ones
- table rows share a width and a column count
- no trailing whitespace, keeping real Markdown hard breaks
- every `.gitbook.yml` redirect target resolves
- a moved or deleted page has a redirect

It reports only findings a change introduces, by running every check
against two trees and diffing. Two consequences:

- Existing debt never fails a build, so a stricter convention can be
  adopted without a repo-wide cleanup first. There are 159 pre-existing
  findings today, mostly heading case.
- Comparing against the **target branch tip** rather than the merge base
  catches a merge that silently undoes a fix already on main. That is
  what happened between #207 and #208: #207 changed a heading level on a
  line #208 had just corrected, from a base that predated it, and git
  reported no conflict.

Finding identity excludes the line number, so moving a paragraph does
not resurface everything below it as new.

Stdlib only, no dependencies, ~2s for the whole repo. Runnable locally:

    python3 .github/scripts/docs_lint.py --base origin/main
The linter needs the target branch to diff against, hence
`fetch-depth: 0` plus an explicit fetch. actions/checkout gives us the
merge of the PR into the target branch, which is what makes the
merge-regression case detectable.

Removes redirect-url-checker.yml. It probed the published site over the
network, which GitBook rate-limits, so it never passed reliably - and it
was `workflow_dispatch` only, so in practice nothing ran on a pull
request at all. Its purpose is covered locally and deterministically by
the new redirect-target and moved-file checks, which read `.gitbook.yml`
and the tree instead of making requests.
The new redirect-target check found 38 redirects aimed at files that no
longer exist, so those old URLs 404 today. They are all fallout from the
reorganisation that introduced `concepts/` and `quick-start/`; only six
distinct targets are involved:

    use-steadybit/actions.md                      -> concepts/actions/README.md                  (25)
    use-steadybit/discovery/README.md             -> concepts/discovery/README.md                  (6)
    getting-started.md                            -> quick-start/getting-started.md                (3)
    troubleshooting/agent.md                      -> troubleshooting/common-fixes/agents.md        (2)
    use-steadybit/experiments/file-import-export  -> .../experiments/share/file-import-export      (1)
    integrate-with-steadybit/webhooks.md          -> integrate-with-steadybit/webhooks/README.md   (1)

Every destination verified to exist. No redirect keys change, so no
currently-working URL is affected.
One place for the rules, so the linter, contributors and any future
review agent agree. Records the decisions that were open:

- Headings are title case, except question-style headings, which stay in
  sentence case - the troubleshooting pages are written as questions and
  title-casing them reads wrong.
- US English: color, behavior, organization, analyze, canceled. This was
  already true throughout, but was never written down, so nothing
  stopped it drifting back.
- Lists use the serial (Oxford) comma. Noted as *not* linted: separating
  a three-item list from a compound such as "open- and closed-source
  extensions" is not reliably detectable, and every sample the draft
  check flagged was a false positive.

Also notes the scope difference the linter applies - British forms are
prose-only, plain misspellings apply everywhere including code samples -
and updates the CI/CD section, which still described the removed
redirect checker.
Review question: the table looked like it mixed wrong spellings with
right ones. It is a `written -> should say` mapping, so "Kubernetes" is
the suggestion for "kuberneters" rather than an entry - but 23 pairs
packed onto continuation lines made that genuinely hard to see. One
entry per line now, aligned and grouped by kind.

Two real problems that question surfaced:

- Lookups were done on a lowercased line, so keys had to be lowercase or
  they would silently never match, and nothing said so. Matching is now
  explicitly case-insensitive, and an assertion rejects a capitalised
  key instead of letting it quietly do nothing.
- A finding quoted the table key rather than the file's own text, so a
  heading reading "Langauge" produced `"langauge" should be ...`.
  It now quotes what is actually written.

Also corrects the suggestion for "langauge" to the common noun
"language"; it only looked capitalised because the single instance we
had sat inside a title.
@ManuelGerding
ManuelGerding merged commit f4e7efd into main Sep 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant