Make the documentation checkable, and close its gaps - #178
Merged
Conversation
Closes #150 `docs/` was already substantial. What it lacked was anything holding it to the code, so this adds the checks first and then fixes what they found — which is the only order that keeps the answer true next month. **`make docs-check` grew two checks.** A documented `make` target that no longer exists is a stale doc in the worst possible place: the first command a new contributor runs. A documented `ICEBERG_*` variable that no setting backs leaves an operator configuring something nothing reads. Both are matched against the Makefile and the settings classes respectively, so neither can drift. The `make X` pattern deliberately only matches code — inline backticks or a line in a fenced block — because matching prose catches "make it clear" and "make the engine", and a check that cries wolf is one people turn off. It also now reads only the files git tracks. `rglob("*.md")` walks a contributor's local tool directories too, which is how it exits non-zero on a permission error in a file that has nothing to do with the project. **`tests/test_docs_invariants.py` checks the claims a regex cannot.** All seven failed when written, which is what made them worth writing: - CONTRIBUTING named `make check` and `make docs-check` but not `lint`, `type`, `test`, `version-check` or `rehearse`. A contributor who runs everything the guide names and still fails CI has been told the wrong thing, and the second time it happens they stop running any of it. It now carries the whole gate as a table, in CI's order, with what each one proves. - Four documents were orphaned — nothing in the repository linked to `retention.md`, `runbooks/controlled-pilot.md`, `spikes/python-3.14-compat.md` or `web/README.md`. A guide nobody links to is the one that rots, because nothing points at it when the thing it describes changes. - ADR 0013 and two runbooks were missing from the index. A runbook is read under pressure by somebody who did not write it; if it is not in the index it will not be found in time to matter. - SECURITY.md answered how to report a vulnerability but not "is my version still getting fixes?" — it now points at the support window, and says fixes are announced as GitHub Security Advisories with the identifier in the CHANGELOG. - The backup/restore runbook claimed a rehearsal without naming one. It now names `make rehearse`, and the test asserts the two reference each other — a runbook that claims to be exercised by a script nobody can find is a claim, not evidence. Against #150's criteria: the production-install and clean-room paths already existed; backup and restore are now literally rehearsed in CI rather than described; security reporting and the support window are public; the contributor gate is CI's gate and a test says so; and the docs are versioned with the release that ships them and checked automatically on every pull request. make check green: 1826 passed.
There was a problem hiding this comment.
Verdict
APPROVE
Completed bounded review across 1 immutable scope(s). Approved: no actionable defects found in the supplied change.
Scope health
Convergence: healthy. Review mode: initial.
Recommended action: CONTINUE_INCREMENTAL.
- No escalation signals.
Prior findings
| Finding | Status |
|---|---|
| — | No prior finding state |
New findings
No new findings.
Fix-induced regressions
- None evidenced.
Uncertainty
- No material uncertainty recorded.
Validation
- Reviewed the supplied immutable diff; exact-head CI is reported as passed.
Residual risks
- None identified.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #150
docs/was already substantial. What it lacked was anything holding it to the code —so this adds the checks first and then fixes what they found, which is the only order
that keeps the answer true next month.
make docs-checkgrew two checksmaketargets existICEBERG_*settings existBoth are matched against the Makefile and the settings classes, so neither can drift.
The
make Xpattern deliberately only matches code — inline backticks, or a linein a fenced block — because matching prose catches "make it clear" and "make the
engine", and a check that cries wolf is one people turn off.
It also now reads only the files git tracks.
rglob("*.md")walks a contributor'slocal tool directories too, which is how it exits non-zero on a permission error in a
file that has nothing to do with the project.
tests/test_docs_invariants.pychecks what a regex cannotAll seven failed when written. That is what made them worth writing:
make checkandmake docs-check, notlint,type,test,version-checkorrehearse. Acontributor who runs everything the guide names and still fails CI has been told
the wrong thing, and the second time it happens they stop running any of it. It now
carries the whole gate as a table, in CI's order, with what each one proves.
retention.md,runbooks/controlled-pilot.md,spikes/python-3.14-compat.md, orweb/README.md. A guide nobody links to is the one that rots, because nothingpoints at it when the thing it describes changes.
pressure by somebody who did not write it; if it is not indexed it will not be found
in time to matter.
It now points at the support window and states that fixes ship as GitHub Security
Advisories with the identifier in the CHANGELOG.
make rehearse, and a test asserts the two reference each other — a runbook thatclaims to be exercised by a script nobody can find is a claim, not evidence.
#150's criteria
runbooks/production-install.md, now reachable and indexedmake checkgreen: 1832 passed.