Skip to content

Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E) - #799

Draft
anth-volk wants to merge 19 commits into
ruff-security-loggingfrom
ruff-structure-imports
Draft

anth-volk wants to merge 19 commits into
ruff-security-loggingfrom
ruff-structure-imports

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #798

Same-Repository Draft PR Check

  • This PR is a draft.
  • This PR is from a branch in PolicyEngine/policyengine-sim-api, not a personal fork.

Summary

Phase 1 step 1.8, group 7: structure and imports. Stacked on #785.

  • Test exemption (decided for this group): **/tests/** and **/authenticated_tests/** now also ignore SLF001, INP001 and PLC0415, alongside S101, D1, ANN and PLR2004. Tests deliberately reach into private members, pytest test directories aren't packages, and tests often import inside a test after patching the environment. Application code still gets every rule. Both test-exemption rules in the registry are updated to match.
  • Rules: ruff.toml gains A, E, FAST, INP, N, PLC and SLF. All 339 findings left after the exemption are fixed or kept with an inline reason.
  • Imports inside functions (PLC0415, 176):
    • Moved (16): only imports that load nothing new, create no cycle and aren't patched by tests. Each was checked by importing every affected module in a fresh interpreter.
    • Kept (160), each with a reason:
      • Modal and image code: 56. Function bodies import packages that exist only in the image.
      • Test seams: 52. Tests monkeypatch these names, so they're looked up at call time.
      • Lazy heavy or optional dependencies: 49.
      • Import cycles: 3.
  • Namespace packages (INP001, 13): .github/scripts, scripts, the executor's scripts and rules/fixtures aren't packages, so they're declared in ruff's top-level namespace-packages setting. Compared with the full ruff check output before the change, nothing else differs. The executor's src/modal/utils scripts keep an inline suppression instead: declaring that directory made ruff treat import modal (the Modal SDK) as first-party, and an __init__.py would change the source copied into the Modal images.
  • Names (N, A, PLC0414, SLF001):
    • Renamed exceptions: the entry app's BackendUnavailableError, BackendTimeoutError and TemporaryStage12UnsupportedRequestError, and the executor's ParityMismatchError. Each is caught by name and never logged or stored by class name.
    • Kept: TemporaryStage12DispatchFailed, because the entry app logs its class name as error_type.
    • Published fields: the 14 camelCase budget-window fields in gateway_models.py keep their names.
    • Other renames: three OpenTelemetry locals, the exit import aliased as app_exit, and four gateway test names shortened for line length.
    • Kept with a reason: the contract's explicit re-exports, the exit object projects import, and two private accesses with no public equivalent.
  • FastAPI (FAST001, 13): response_model arguments that repeat the return type are dropped. The gateway golden OpenAPI is unchanged, and the entry, executor and gateway OpenAPI dumps are identical before and after.
  • Line length (E501): wrapped, with every split string's text identical, which a syntax-tree comparison confirms. Four multi-line fake shell scripts in tests keep a suppression, because their one-line printf JSON can't be split without changing the script.
  • Modal images: many executor and gateway image files change, mostly in import placement and line wrapping; src/modal/app.py gains no __future__ import. CI's image smoke will run when this PR reaches the bottom of the stack.
  • Registry: sim.quality.ruff-lint now lists the seven families. The shared rules tool is synced from the Public API's group 7 (Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E) policyengine-api#3920), so it's byte-identical again (tool sha256 f7e9b6462761).

Testing

  • All seven projects' test suites pass, with the same counts as the base:

    Project Passed
    fastapi 14
    contract 145
    observability 30
    stage12 12
    entry 224
    gateway 214
    executor 744
  • The gateway golden OpenAPI test passes, and the golden file is unchanged.

  • ruff check (0.15.22) is clean repo-wide, and ruff format --check is unchanged.

  • The rules guard, the ci-scripts guard and the rules tool unittest pass.

  • The integration-test project's conftest change was checked by parsing only; its tests need live services.

  • CI/CD: no workflow changed, and no environment variable was added or rewired.

  • CI: this PR is above the bottom of its stack, so it runs only CI complete, which fails and says why (step 0.7, Make CI enforcement trustworthy before the rules registry #754). Its full run starts once the PRs below it merge.

🤖 Generated with Claude Code

@anth-volk
anth-volk added this pull request to stack #755 October 14, 2026 13:54
@anth-volk anth-volk changed the title ruff structure imports Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E) Oct 14, 2026
@anth-volk
anth-volk force-pushed the ruff-structure-imports branch 2 times, most recently from 2a7ddcc to 30b23b7 Compare October 14, 2026 14:47
@anth-volk
anth-volk force-pushed the ruff-structure-imports branch 4 times, most recently from b013982 to 548762d Compare October 14, 2026 20:04
@anth-volk
anth-volk force-pushed the ruff-structure-imports branch 2 times, most recently from 20049c1 to 68eea13 Compare October 14, 2026 21:43
anth-volk and others added 16 commits October 15, 2026 01:52
Tests deliberately reach into private members, pytest test directories
aren't packages, and tests often import inside a test after patching the
environment. Application code still gets every rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FastAPI takes the response model from the return annotation when
response_model is omitted, so the generated OpenAPI documents are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Alias the shared exit registry as app_exit where it is imported, and keep
  the policyengine_fastapi.exit.exit name that projects import.
- Use snake_case for the OpenTelemetry provider and log-handler locals.
- Add the Error suffix to exceptions that never leave their project:
  BackendUnavailableError, BackendTimeoutError,
  TemporaryStage12UnsupportedRequestError and ParityMismatchError.
  TemporaryStage12DispatchFailed keeps its name, which the entry app logs
  as error_type.
- Keep the camelCase budget-window fields, which are published JSON, the
  explicit re-exports in stage12_execution, and the two deliberate private
  accesses, each with a reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Declare the script directories (.github/scripts, scripts and the executor's
scripts) and the rules proof fixtures as namespace packages in ruff.toml.
Ruff's full output, with every rule selected, is otherwise unchanged by the
setting.

The executor's src/modal/utils scripts keep an inline suppression instead:
declaring that directory, or adding an __init__.py, makes ruff treat it as
part of a first-party "modal" package and regroup its Modal SDK imports.
An __init__.py would also change the src.modal source baked into the Modal
images.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…(ruff PLC0415)

Each deferred import keeps an inline reason: the GCP exporters, logging
instrumentation and metadata probe load only when used, and only the
bundle-versions command runs where policyengine is installed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
app.py already imports stage12_backend at module level for its exception
types, so binding Stage12ComparisonBackend there too loads nothing new.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bind SPMSelection in endpoints.py's existing policyengine_simulation_contract.spm
import. Keep the Modal function body's imports, which build the web app in
its container, and the smoke app's in-image imports, each with a reason.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Move only imports that load nothing new and that no test patches:
pathlib and importlib, the policyengine_simulation_contract helpers, names
from a module the file already imports at the top, and RegionGroupStrategy,
which policyengine.core already loads.

Every other deferred import keeps an inline reason: Modal function bodies and
the image-build and smoke functions, lazily loaded policyengine models and
heavy libraries, call-time resolution that the tests' patch seams rely on,
and import cycles in the runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Split long messages and the GCP log format with implicit string
concatenation, so every string is unchanged, and rewrap two docstrings that
no published schema uses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bind the long generated endpoint functions to shorter local names before
calling them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Split long strings with implicit concatenation, so every value is unchanged.
The fake curl and gcloud scripts keep their single-line printf output with a
suppression on the string.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Split long messages and test strings with implicit concatenation, so every
value is unchanged, and shorten docstrings that no published schema uses.
The fake gh script keeps its single-line JSON output with a suppression on
the string.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Split the Modal app's FastAPI description with implicit concatenation, so
the published text is unchanged, and shorten four test names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Select A, E, FAST, INP, N, PLC and SLF. Every finding outside the shared
scripts/rules tool is fixed or carries an inline reason; the rules tool's
findings are fixed upstream and synced separately.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
sim.quality.ruff-lint now lists A, E, FAST, INP, N, PLC and SLF.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anth-volk and others added 3 commits October 15, 2026 01:52
The shared scripts/rules tool takes the Public API's group 7 fixes:
wrapped long lines, BaseUnavailableError, and assert_violation in its
tests. The rendered rules page picks up the tool's new hash.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#743 and #744 landed on main after this stack branched. Group 7, which
this PR enables, flags their new code; each fix follows this PR's
conventions:

- E501: wrap the long strings in their routing, sentinel, gateway and
  test code; every message and expected value reads the same.
- N818: RoutingConflict never leaves the executor, so it becomes
  RoutingConflictError, as this PR renames the other in-project errors.
- INP001: the sentinel script in src/modal/utils keeps the inline reason
  the other scripts there carry.
- PLC0415: the sentinel's deferred contract and runtime imports and the
  package updater's optional receipt import keep inline reasons.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#789 landed on main after this stack branched. E501, which this PR
enables, flags the signature line of its lost-commit coordinator test,
whose name cannot be wrapped; the line keeps a reasoned suppression
rather than a shorter, vaguer name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk force-pushed the ruff-structure-imports branch from 68eea13 to 51e611a Compare October 14, 2026 21:52

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable ruff structure and import rules (INP, PLC, SLF, N, A, FAST, E)

1 participant