Skip to content

Add a rules registry that makes every enforced and unenforced rule visible - #752

Draft
anth-volk wants to merge 6 commits into
phase-0-enforcementfrom
rules-registry
Draft

anth-volk wants to merge 6 commits into
phase-0-enforcementfrom
rules-registry

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #751

Same-Repository Draft PR Check

  • This PR is a draft.
  • This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

Stacked PR. This is the middle of a native GitHub stack: Phase 0 (#754) → this PR → steps 1.3, 1.5 and 1.6 (#757). It's based on phase-0-enforcement and was rebased onto it, so the diff shows only this layer.

  • Adds rules/registry.toml: every rule that governs this repository, with its tier (mechanical, metric or judgment), status (enforced, ratcheted, advisory or doc-only), enforcer, and the CI job that runs it. 38 rules, of which 13 are doc-only gaps, for example the gateway/executor import boundary and the observability rules. After Phase 0, the image-smoke and test-matrix rules describe the fixed state; nothing enforces them yet. Phase 0's explicit ruff rule list is registered as an advisory lint rule. Code-owned rule machinery and the agent no-bypass rule are registered as doc-only until branch rules enforce them.

  • Adds scripts/rules/, a standard-library tool with validate, render and guard commands. The guard fails when:

    • an enforcer doesn't exist;
    • a CI job doesn't run the enforcer it claims to;
    • a guard test, quality guard or CI job isn't registered;
    • a proof test isn't collected by any CI job;
    • a baseline grows relative to the base branch;
    • an exception or review_by date expires;
    • the rendered views are stale.

    The tool is byte-identical with the same directory in policyengine-api; see scripts/rules/SYNC.md.

  • Renders docs/generated/rules.md (a summary, a Gaps section, and a per-category table with Mermaid graphs), and a generated Rules block in AGENTS.md.

  • Adds scripts/run_quality_guards.py and a dependency-free quality-guards PR job. The job runs the tool's unittest suite and the guards, with fetch-depth: 0 so baselines can be compared with the base branch.

  • Adds a proof fixture and test for sim.stage12.no-schema-ddl, showing the DDL check rejects a bad example and accepts a good one.

  • Documents how to add or change a rule in docs/engineering/skills/rules-registry.md.

Ruff rules are never ratcheted: adopting a ruff family is one standalone PR that fixes every violation.

Testing

  • python -m unittest discover -s scripts/rules/tests -t .: 54 tests pass. The first version, with 50, was run on Python 3.11 to 3.14; the 4 added since cover proof tests that must run in CI.
  • python scripts/run_quality_guards.py: rules-registry: passed.
  • python -m scripts.rules guard --base-ref origin/main: passes. The base has no registry yet, so the baseline comparison is skipped.
  • projects/policyengine-simulation-entry: uv run pytest tests/ passes (159 tests, including the new proof test).
  • libs/policyengine-stage12-persistence: uv run pytest tests/ passes (12 tests).
  • ruff format --check passes on all src directories and the new files.
  • Each of these negative checks failed the guard as expected and was then reverted:
    • removing a deployment test from a rule;
    • adding an unregistered GUARDS entry;
    • hand-editing the AGENTS block;
    • changing a job invocation to a nonsense value;
    • adding an expired exception;
    • adding a baseline to a ruff rule;
    • removing a project from the test matrix.

🤖 Generated with Claude Code

anth-volk and others added 6 commits October 12, 2026 23:28
Standard-library tool (scripts/rules) that loads rules/registry.toml, checks
each rule's enforcer and CI wiring, finds unregistered checks, compares
ratchet baselines with the base branch, and renders the human and agent
views. Byte-identical with policyengine-api; see scripts/rules/SYNC.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a quality guard runner and a dependency-free quality-guards PR job that
runs the tool's unittest suite and the guards, with full history so
baselines can be compared with the base branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Registers the Stage 12, deployment-asset and OpenAPI golden tests, ruff
format and CI jobs, plus rules stated only in docs as doc-only. Adds a proof
fixture and test for sim.stage12.no-schema-ddl.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sync scripts/rules with policyengine-api: the guard now checks that each
proof test is collected by a CI job, not only that it exists. A
pytest-enforced rule's proof runs in the rule's own job; other rules name
the job with proof.workflow, proof.job and proof.invocation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Register the explicit ruff rule selection in ruff.toml as an advisory lint
rule, update the image-smoke and test-matrix rules now that Phase 0 fixed
both gaps, and register the code-owned rule machinery and the agent
no-bypass rule as doc-only until branch rules enforce them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk changed the base branch from main to phase-0-enforcement October 12, 2026 19:29
@anth-volk
anth-volk added this pull request to stack #755 October 12, 2026 19:29

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a rules registry that makes every enforced and unenforced rule visible

1 participant