Skip to content

Make the documentation checkable, and close its gaps - #178

Merged
icebergai-review-bot[bot] merged 1 commit into
mainfrom
docs-v2
Aug 16, 2026
Merged

Make the documentation checkable, and close its gaps#178
icebergai-review-bot[bot] merged 1 commit into
mainfrom
docs-v2

Conversation

@richardmhope

Copy link
Copy Markdown
Contributor

Closes #150

docs/ was already substantial. What it lacked was anything holding it to the code —
so this adds the checks first and then fixes what they found, which is the only order
that keeps the answer true next month.

make docs-check grew two checks

Check The rot it catches
Documented make targets exist a renamed target leaves an instruction that fails on the first command a new contributor runs
Documented ICEBERG_* settings exist a renamed field leaves an operator configuring something nothing reads

Both are matched against the Makefile and the settings classes, so neither can drift.
The make X pattern deliberately only matches code — inline backticks, or a line
in a fenced block — because matching prose catches "make it clear" and "make the
engine", and a check that cries wolf is one people turn off.

It also now reads only the files git tracks. rglob("*.md") walks a contributor's
local tool directories too, which is how it exits non-zero on a permission error in a
file that has nothing to do with the project.

tests/test_docs_invariants.py checks what a regex cannot

All seven failed when written. That is what made them worth writing:

  • CONTRIBUTING named two of CI's five gates. It listed make check and
    make docs-check, not lint, type, test, version-check or rehearse. A
    contributor who runs everything the guide names and still fails CI has been told
    the wrong thing, and the second time it happens they stop running any of it. It now
    carries the whole gate as a table, in CI's order, with what each one proves.
  • Four documents were orphaned. Nothing in the repository linked to
    retention.md, runbooks/controlled-pilot.md, spikes/python-3.14-compat.md, or
    web/README.md. A guide nobody links to is the one that rots, because nothing
    points at it when the thing it describes changes.
  • ADR 0013 and two runbooks were missing from the index. A runbook is read under
    pressure by somebody who did not write it; if it is not indexed it will not be found
    in time to matter.
  • SECURITY.md answered how to report but not "is my version still getting fixes?"
    It now points at the support window and states that fixes ship as GitHub Security
    Advisories with the identifier in the CHANGELOG.
  • The backup/restore runbook claimed a rehearsal without naming one. It now names
    make rehearse, and a test asserts the two reference each other — a runbook that
    claims to be exercised by a script nobody can find is a claim, not evidence.

#150's criteria

  • A fresh operator can complete a production-oriented installation — runbooks/production-install.md, now reachable and indexed
  • Backup and restore instructions are rehearsed — literally, in CI, since Rehearse the upgrade, rollback, and recovery paths in CI #177
  • Security reporting and response expectations are public — plus the support window
  • Contributor checks match CI behavior — and a test says so
  • Docs are versioned with releases and checked automatically — they ship in the tag, and every PR checks links, targets, and settings

make check green: 1832 passed.

Closes #150

`docs/` was already substantial. What it lacked was anything holding it to the
code, so this adds the checks first and then fixes what they found — which is the
only order that keeps the answer true next month.

**`make docs-check` grew two checks.** A documented `make` target that no longer
exists is a stale doc in the worst possible place: the first command a new
contributor runs. A documented `ICEBERG_*` variable that no setting backs leaves
an operator configuring something nothing reads. Both are matched against the
Makefile and the settings classes respectively, so neither can drift. The
`make X` pattern deliberately only matches code — inline backticks or a line in a
fenced block — because matching prose catches "make it clear" and "make the
engine", and a check that cries wolf is one people turn off.

It also now reads only the files git tracks. `rglob("*.md")` walks a
contributor's local tool directories too, which is how it exits non-zero on a
permission error in a file that has nothing to do with the project.

**`tests/test_docs_invariants.py` checks the claims a regex cannot.** All seven
failed when written, which is what made them worth writing:

- CONTRIBUTING named `make check` and `make docs-check` but not `lint`, `type`,
  `test`, `version-check` or `rehearse`. A contributor who runs everything the
  guide names and still fails CI has been told the wrong thing, and the second
  time it happens they stop running any of it. It now carries the whole gate as a
  table, in CI's order, with what each one proves.
- Four documents were orphaned — nothing in the repository linked to
  `retention.md`, `runbooks/controlled-pilot.md`, `spikes/python-3.14-compat.md`
  or `web/README.md`. A guide nobody links to is the one that rots, because
  nothing points at it when the thing it describes changes.
- ADR 0013 and two runbooks were missing from the index. A runbook is read under
  pressure by somebody who did not write it; if it is not in the index it will
  not be found in time to matter.
- SECURITY.md answered how to report a vulnerability but not "is my version still
  getting fixes?" — it now points at the support window, and says fixes are
  announced as GitHub Security Advisories with the identifier in the CHANGELOG.
- The backup/restore runbook claimed a rehearsal without naming one. It now names
  `make rehearse`, and the test asserts the two reference each other — a runbook
  that claims to be exercised by a script nobody can find is a claim, not
  evidence.

Against #150's criteria: the production-install and clean-room paths already
existed; backup and restore are now literally rehearsed in CI rather than
described; security reporting and the support window are public; the contributor
gate is CI's gate and a test says so; and the docs are versioned with the release
that ships them and checked automatically on every pull request.

make check green: 1826 passed.

@icebergai-review-bot icebergai-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict

APPROVE

Completed bounded review across 1 immutable scope(s). Approved: no actionable defects found in the supplied change.

Scope health

Convergence: healthy. Review mode: initial.
Recommended action: CONTINUE_INCREMENTAL.

  • No escalation signals.

Prior findings

Finding Status
No prior finding state

New findings

No new findings.

Fix-induced regressions

  • None evidenced.

Uncertainty

  • No material uncertainty recorded.

Validation

  • Reviewed the supplied immutable diff; exact-head CI is reported as passed.

Residual risks

  • None identified.

@icebergai-review-bot
icebergai-review-bot Bot merged commit c6d77a8 into main Aug 16, 2026
6 checks passed
@icebergai-review-bot
icebergai-review-bot Bot deleted the docs-v2 branch August 16, 2026 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish operator, security, and contribution documentation

1 participant