Add a rules registry that makes every enforced and unenforced rule visible

.github/workflows/pr.yml

@@ -16,7 +16,7 @@ jobs:
     # every job passed. See docs/engineering/skills/github-prs.md.
     name: CI complete
     if: ${{ always() }}
-    needs: [test, lint, docker-build, test-integration]
+    needs: [test, lint, quality-guards, docker-build, test-integration]
     runs-on: ubuntu-latest
     permissions:
       contents: read
@@ -107,6 +107,29 @@ jobs:
           fi
         done
 
+  quality-guards:
+    name: Quality guards
+    # Full CI runs only on the lowest unmerged pull request in a stack.
+    if: ${{ !github.event.pull_request.stack || github.event.pull_request.stack.base.ref == github.event.pull_request.base.ref }}
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v6
+      with:
+        # The rules registry guard compares baselines with the base branch.
+        fetch-depth: 0
+
+    - name: Set up Python
+      uses: actions/setup-python@v6
+      with:
+        python-version: '3.13'
+
+    - name: Test the rules registry tool
+      run: python -m unittest discover -s scripts/rules/tests -t .
+
+    - name: Run quality guards
+      run: python scripts/run_quality_guards.py
+
   docker-build:
     name: Docker build
     needs: [test, lint]  # Only run if tests and linting pass

AGENTS.md

@@ -18,6 +18,10 @@ Before changing correlation identifiers, telemetry transport, spans, stage
 names, logging, or observability failure handling, read
 `docs/engineering/skills/observability.md`.
 
+When adding, changing, or removing a rule, a guard test, a checker
+configuration, or a CI job that enforces one, read
+`docs/engineering/skills/rules-registry.md`.
+
 ## Development
 
 - Use Python 3.13 and `uv`.
@@ -59,6 +63,77 @@ same-repository draft PR.
 
 - The simulation service lives in `projects/policyengine-simulation-executor`.
 - API integration tests live in `projects/policyengine-apis-integ`.
-- PR CI runs simulation unit tests, Ruff format checks, Docker build, and local
-  integration tests.
+- PR CI runs simulation unit tests, Ruff format checks, quality guards
+  (including the rules registry), Docker build, and local integration tests.
 - There is currently no repository-wide changelog fragment requirement.
+
+## Rules
+
+<!-- BEGIN GENERATED: rules (python -m scripts.rules render) -->
+
+Every rule for this repository lives in `rules/registry.toml` and is rendered,
+with its enforcement and gaps, in `docs/generated/rules.md`. Blocking rules
+fail CI. Doc-only rules are not checked automatically, so follow them yourself.
+To add or change a rule, read `docs/engineering/skills/rules-registry.md`.
+
+### Service and package boundaries
+
+- `sim.boundaries.gateway-executor-decoupled`: The gateway and executor never import each other; they communicate only through the models and state helpers in policyengine-simulation-contract. (doc-only, not checked: follow it yourself)
+- `sim.boundaries.unit-tests-need-no-services`: Unit tests never require real network credentials, Modal, Hugging Face, GCP or deployed services; those seams are mocked. (doc-only, not checked: follow it yourself)
+
+### Stage 12 persistence
+
+- `sim.stage12.deployment-scope`: Main-branch deploys roll out Stage 12 in both environments; a Stage 12-only deploy runs candidate tests and the requested promotion but never redeploys existing Modal resources; the Stage 12 worker can request a Google OIDC token. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.stage12.deployment-uses-api-schema`: Stage 12 deployment uses the API-owned schema directly, and the Stage 12 runtime it deploys uses typed DML without raw SQL or schema changes. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.stage12.no-migration-chain`: This repository has no schema migration chain (no Alembic); Stage 12 tables are migrated by policyengine-api. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_stage12_schema_authority.py`)
+- `sim.stage12.no-production-user-associations`: Stage 12 runtime modules never create production user associations (user_simulations or user_reports). (blocking: pytest `projects/policyengine-simulation-entry/tests/test_stage12_schema_authority.py`)
+- `sim.stage12.no-schema-ddl`: Simulation runtimes and the Stage 12 persistence library contain no schema DDL (CREATE, ALTER, DROP or TRUNCATE of tables, types, schemas or databases, or create_all); policyengine-api owns the schema. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_stage12_schema_authority.py`)
+- `sim.stage12.persistence-typed-dml-only`: The Stage 12 persistence library and runtime use typed SQLAlchemy Core DML only: no raw SQL text, driver-level SQL or schema operations. (blocking: pytest `libs/policyengine-stage12-persistence/tests/test_source_boundaries.py`)
+- `sim.stage12.production-lock-has-db-runtime`: The production lock file includes the direct Stage 12 database runtime dependencies. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+
+### Deployment pipeline
+
+- `sim.deploy.container-context-minimal`: Container builds exclude local environments and unrelated projects, and the entrypoint container builds for the requested target platform. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.deploy.docker-images-build`: The simulation entry and executor Docker images build. (blocking: CI job `docker-build`)
+- `sim.deploy.gcloud-without-terraform`: Deployment uses the gcloud-based workflow, not Terraform. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.deploy.image-smoke-covers-every-image-lib`: The image smoke path filter includes every library mounted into a Modal image, so a change to any of them runs the smoke. (doc-only, not checked: follow it yourself)
+- `sim.deploy.modal-image-smoke`: App entrypoints import inside the real Modal images before merge, whenever a library baked into an image changes. (blocking: CI job `image-smoke`)
+- `sim.deploy.promotion-order-explicit`: Full-stack promotion order is explicit, and the reusable deployment rejects unexpectedly skipped phases. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.deploy.revision-tags-valid`: Cloud Run revision tags are valid on a workflow's first run and fit the existing service names. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.deploy.shell-scripts-valid`: Deployment shell scripts have valid shell syntax. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.deploy.smoke-tests-strict`: Candidate and stable smoke tests reject a revision mismatch without retrying, and fail if the previous revision does not converge. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.deploy.traffic-changes-exact`: Traffic changes target an exact, validated revision, are reversible, and refuse to run over an intervening promotion. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.deploy.workflow-shell-safety`: Cloud Run deployment never interpolates GitHub expressions into shell, escapes environment values, and pins secret versions. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+
+### API contracts and integration
+
+- `sim.contracts.clients-publish-deployed-commit`: Client publication checks out the commit that was actually deployed. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.contracts.full-integration-beta-only`: The complete integration suite runs against beta deployments only, never production. (blocking: pytest `projects/policyengine-simulation-entry/tests/test_deployment_assets.py`)
+- `sim.contracts.gateway-openapi-golden`: The gateway's OpenAPI spec matches the checked-in golden file; regenerate it deliberately when the API changes. (blocking: pytest `projects/policyengine-simulation-gateway/tests/test_openapi_golden.py`)
+- `sim.contracts.local-integration`: Generated-client integration tests pass against the locally built services. (blocking: CI job `test-integration`)
+- `sim.contracts.regenerate-clients`: When public API schemas change, regenerate the clients and run the generated-client integration tests. (doc-only, not checked: follow it yourself)
+
+### Observability
+
+- `sim.observability.failures-never-break-requests`: Invalid observability configuration fails at build or deploy time, and telemetry failures at runtime never fail a request or computation. (doc-only, not checked: follow it yourself)
+- `sim.observability.header-transport-only`: observability_id travels only in the X-PolicyEngine-Observability-Id header, never in a simulation JSON body, and polling never invents one. (doc-only, not checked: follow it yourself)
+- `sim.observability.id-is-diagnostic-only`: observability_id has diagnostic meaning only: never use it for idempotency, authorization, database identity, routing or business logic. (doc-only, not checked: follow it yourself)
+- `sim.observability.registered-stages-only`: Span names come from the stage plans in policyengine_simulation_observability.stages; runtime code never adds span-name string literals. (doc-only, not checked: follow it yourself)
+
+### Code quality and tests
+
+- `sim.quality.every-project-tested`: Every project and library with unit tests runs them in the PR test matrix. (doc-only, not checked: follow it yourself)
+- `sim.quality.ruff-format-src`: Source code under projects/*/src and libs/*/src is formatted with ruff format. (blocking: ruff format)
+- `sim.quality.ruff-format-tests`: Test code is formatted with ruff format too; CI currently checks only src directories. (doc-only, not checked: follow it yourself)
+- `sim.quality.ruff-lint`: Python code passes ruff check with the rules selected in ruff.toml (ruff's defaults, made explicit); CI does not run ruff check yet. (advisory: ruff check (E4, E7, E9, F), not blocking)
+- `sim.quality.tests-pass`: Each project's unit tests pass in its own locked environment. (blocking: CI job `test`)
+
+### Process and repository hygiene
+
+- `sim.process.agents-never-bypass`: Agents never bypass branch rules, even with admin credentials: no gh pr merge --admin and no bypass option; a blocked merge is reported to a human. (doc-only, not checked: follow it yourself)
+- `sim.process.rule-machinery-code-owned`: Changes to the rule machinery (rules/, scripts/rules/, scripts/run_quality_guards.py and CODEOWNERS) need approval from a core developer other than the author. (doc-only, not checked: follow it yourself)
+- `sim.process.rules-registry`: Every rule is registered in rules/registry.toml, is enforced where it claims to be, and its rendered views are current. (blocking: quality guard `rules-registry`)
+- `sim.process.same-repo-draft-prs`: PRs are opened as drafts from branches in PolicyEngine/policyengine-sim-api, never from forks, and start with 'Fixes #<issue>'. (doc-only, not checked: follow it yourself)
+- `sim.process.stacked-prs-full-ci-at-bottom`: In a stack of pull requests, only the lowest unmerged one runs the full PR workflow and the Modal image smoke. Every pr.yml job is needed by the CI complete gate, which fails on the pull requests above it and otherwise passes only when every job passed, so a stack merges bottom-up and nothing merges untested. (blocking: pytest `projects/policyengine-simulation-executor/tests/test_ci_complete_gate.py`)
+
+<!-- END GENERATED: rules -->

changelog_entry.yaml

@@ -1,5 +1,7 @@
 - bump: patch
   changes:
+    added:
+      - Added a rules registry (rules/registry.toml) with a standard-library tool that validates it, checks every rule is enforced where it claims, and renders docs/generated/rules.md and the Rules block in AGENTS.md
     changed:
       - Added CODEOWNERS for the rule machinery, pinned CI's ruff to 0.15.22 with an explicit rule selection, and documented that agents never bypass branch rules
       - Ran the Modal image smoke for Stage 12 persistence changes and added policyengine-fastapi to the PR test matrix

docs/engineering/skills/README.md

@@ -16,3 +16,5 @@ Current skills:
 - `observability.md`: identifier ownership, HTTP and Modal transport,
   persistence, registered runtime stages, trace boundaries, and failure
   isolation.
+- `rules-registry.md`: how rules are registered in `rules/registry.toml`,
+  enforced, proven, and rendered into `docs/generated/rules.md`.

docs/engineering/skills/rules-registry.md

@@ -0,0 +1,147 @@
+# Rules Registry
+
+Read this before adding, changing, relaxing or removing a rule, a guard test, a
+checker configuration (ruff, mypy, Pyright, Import Linter), or a CI job that
+enforces one.
+
+Every rule that governs this repository lives in `rules/registry.toml`. The
+registry says what each rule requires, how mechanically it is enforced, which
+CI job runs it, and whether its checker has been proven to reject a bad example.
+`python -m scripts.rules render` turns it into `docs/generated/rules.md` and the
+Rules block in `AGENTS.md`; the `rules-registry` quality guard fails CI when the
+registry, the checkers and the rendered views disagree.
+
+## Commands
+
+```bash
+python -m scripts.rules validate                    # schema and file checks only
+python -m scripts.rules render                      # rewrite rules.md and the AGENTS.md block
+python -m scripts.rules guard --base-ref origin/<default-branch>   # everything CI checks
+python -m unittest discover -s scripts/rules/tests -t .            # the tool's own tests
+```
+
+All commands use only the standard library, so they run without installing the
+project.
+
+## Adding or changing a rule
+
+A PR that adds, tightens or relaxes a rule contains all of these:
+
+1. **The check**: the checker, config or test change.
+2. **A registry entry** (fields below).
+3. **Proof, appropriate to the tier**: see "Proof".
+4. **Zero violations, or a baseline** where the baseline policy allows one.
+5. **CI wiring**: a `[[rule.ci]]` reference to the job that runs the check.
+6. **Doc update**: explanatory skills docs cite the rule id instead of
+   restating the rule as unchecked prose.
+7. **Regenerated outputs**: run `python -m scripts.rules render` and commit
+   `docs/generated/rules.md` and `AGENTS.md`.
+
+If a rule exists only in prose, register it with `status = "doc-only"` so the
+gap is visible, rather than leaving it out.
+
+## Fields
+
+| Field | Meaning |
+| --- | --- |
+| `id` | Lowercase dotted id, e.g. `api.v2.services-no-sql`. Never reuse an id. |
+| `category` | One of the `[[category]]` ids; controls grouping in the rendered views. |
+| `statement` | One plain-English sentence an agent can follow. |
+| `rationale` | Path (optionally `#anchor`) to the doc that explains the rule. |
+| `scope` | Globs for the files the rule constrains; each must match a file. |
+| `tier` | How mechanically the rule can be checked (see "Tiers"). |
+| `status` | `enforced`, `ratcheted`, `advisory` or `doc-only`. |
+| `autofix` | Whether the enforcer can fix violations itself (e.g. ruff format). |
+| `review_by` | Optional date; the guard fails once it passes. |
+| `[rule.enforcer]` | What checks the rule; `kind` plus kind-specific fields. |
+| `[[rule.ci]]` | `workflow`, `job`, and an `invocation` string that must appear in that job's `run:` text (or its reusable-workflow `uses:`). |
+| `[rule.proof]` | Optional; see "Proof". |
+| `[rule.baseline]` | Only for `ratcheted` rules: `file` listing tolerated violations, and their `count`. |
+| `[[rule.exception]]` | Temporary carve-out: `path`, `reason`, `expires`. |
+| `[rule.downgrade]` | Required `reason` when a rule moves from enforced/ratcheted to advisory/doc-only. |
+
+`blocking` is derived, never declared: a rule blocks merges when its status is
+`enforced` or `ratcheted` and its tier is not `judgment`.
+
+### Enforcer kinds
+
+| Kind | Fields | Resolved by the guard |
+| --- | --- | --- |
+| `pytest` | `nodes` (`path::test` or `path::Class::test`), optional `project` (sub-project directory the paths are relative to) | Each test exists, and the CI job's command collects it |
+| `quality-guard` | `name`, optional `runner` | The name is in `GUARDS` in `scripts/run_quality_guards.py` |
+| `ruff` | `mode` = `format`, `lint` (with `codes`) or `per-file-ignore` (with `pattern`, `codes`) | Codes are selected / the ignore entry exists |
+| `mypy` | `paths` | Each path is in `[tool.mypy].files` |
+| `workflow-job` | optional `step` | The CI job exists (the job itself is the check) |
+| `pyright`, `import-linter` | `config` (+ `contract` for Import Linter) | The config file exists |
+| `docs-guard` | `doc`, `marker` | The doc exists |
+| `llm-review` | `prompt`, `model` | The prompt exists; judgment tier only |
+| `none` | | Nothing; only for `doc-only` and `advisory` rules |
+
+## Tiers
+
+| Tier | Covers | Can block? | Proof |
+| --- | --- | --- | --- |
+| `mechanical` | A pattern is present or not | Yes | A bad fixture is rejected and a good fixture is accepted |
+| `metric` | A number crosses a threshold | Yes | Fixtures just below and just above the threshold |
+| `judgment` | Needs reading comprehension (LLM review plus human) | No | A labelled eval set with measured precision |
+
+Judgment rules graduate: when the same judgment finding recurs, turn it into a
+mechanical rule.
+
+## Proof
+
+A proof shows the checker actually rejects what the rule forbids. Put fixtures
+under `rules/fixtures/<rule-id>/` and add a test named `test_rule_proof_<name>`
+that runs the rule's checker on them.
+
+| Tier | Proof fields |
+| --- | --- |
+| `mechanical` | `bad_fixture`, `good_fixture`, `test` (+ optional `project`) |
+| `metric` | `below_fixture`, `above_fixture`, `threshold`, `test` |
+| `judgment` | `eval_set`, `min_precision`, `measured_precision`, `measured_on` |
+
+The proof test asserts that the bad fixture is rejected and the good one
+accepted. The guard doesn't run it; it checks that the fixtures exist and that a
+CI job collects the proof test:
+
+- For a `pytest`-enforced rule, the proof test must be collected by the rule's
+  own `[[rule.ci]]` job (for example, under the paths `make test` runs).
+- For any other enforcer, name the job that runs the proof test with
+  `proof.workflow`, `proof.job` and `proof.invocation` (all three or none).
+
+Where possible, make the bad fixture a real past mistake from git history.
+Blocking rules without proof are listed as "unproven" in `rules.md`; they don't
+fail the guard yet.
+
+## Baselines
+
+- **Ruff is never ratcheted.** Adopting a ruff rule family is one standalone PR
+  that changes the select list and fixes every resulting violation. Don't use
+  `--add-noqa` or blanket per-file ignores.
+- Other enforcers (mypy, Pyright, Import Linter, custom guards) may start
+  `ratcheted` with a baseline file of tolerated violations. Baselines may only
+  shrink: the guard compares counts against the base branch.
+- Moving a rule from enforced/ratcheted to advisory/doc-only needs an explicit
+  `[rule.downgrade] reason`.
+
+## Hidden rules
+
+The guard fails when something enforces a rule that the registry doesn't
+describe:
+
+- a `GUARDS` entry in `scripts/run_quality_guards.py`;
+- a ruff format check in CI, an explicitly selected ruff code, or a ruff
+  per-file ignore;
+- a `[tool.mypy].files` entry;
+- a test matching `[discovery].guard_test_globs` or listed in
+  `extra_guard_tests`;
+- a job in a `[discovery].workflows` file.
+
+Register it, or list it under `discovery.ignore_jobs` / `discovery.ignore_tests`
+with a reason.
+
+## Keeping the tool in sync
+
+`scripts/rules/` is byte-identical in policyengine-api and policyengine-sim-api;
+see `scripts/rules/SYNC.md`. Change it in both repositories together and bump
+`TOOL_VERSION` and each registry's `tool_version`.