Skip to content

Refactor the economy and calculation services within ruff's complexity limits - #3932

Draft
anth-volk wants to merge 10 commits into
ruff-idioms-argumentsfrom
ruff-complexity-economy
Draft

anth-volk wants to merge 10 commits into
ruff-idioms-argumentsfrom
ruff-complexity-economy

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #3929

Summary

Phase 1 step 1.8, group 8b, part 1 of 3: the economy and calculation services, refactored within ruff's complexity limits. Stacked on #3924. Part 2 (#3933) covers the rest of the application, and part 3 (#3934) covers tests, scripts and the rules tool, then switches ruff to ALL.

  • All 72 findings in scope are fixed by refactoring, with no suppressions. Before each refactor, a commit adds characterization tests that pin current behaviour; 166 in all.
  • Economy comparison (economy_comparison.py):
    • The UK constituency and local-authority breakdowns each had complexity 22 and 57 statements. They keep their public signatures and now share one breakdown loop, with helper tables for income-change buckets and nation prefixes; each keeps its own nation-name casing.
    • Poverty age thresholds become ADULT_MIN_AGE and SENIOR_MIN_AGE.
    • Published results are pinned: tests/data/economy_comparison_snapshots.json stores the complete output of compare_economic_outputs and both breakdowns for UK, US and other-country inputs (17 regions each). It records every dict's key order, floats as repr, value types and the logs, and the refactor reproduces it exactly. Regenerate it only deliberately, with ECONOMY_COMPARISON_SNAPSHOTS=update. The file is about 200 KB.
  • Economy service:
    • get_economic_impact and get_budget_window_economic_impact take an EconomicImpactRequest, in its own module, instead of 9–11 arguments.
    • The impact and budget-window flows are split into named steps: reuse, stored answer, create, claim, replay, poll and submit.
    • The private option builders take one setup-options object.
  • Reform-impact cache and service: ReformImpactScope, ReformImpactIdentity and ReformImpactExecution replace the long argument lists. The cache's key strings and stored records are unchanged, and pinned by tests. set_error_reform_impact and set_complete_reform_impact drop the scope arguments they already ignored.
  • Other modules:
    • budget_window_cache.build_key takes the scope.
    • HouseholdCalculationCache.get splits into a well-formedness check and a receipt check.
    • spm.normalize_spm_selection and worker_spm._check_worker_result are split into steps, with the same error codes, messages and causes.
    • household_calculation_service takes a HouseholdCalculationOptions.
  • Call sites: routes/economy_routes.py, routes/household_routes.py and the affected tests.

Testing

  • pytest tests/unit tests/contract, to_refactor and the dedupe integration test (the make test set) at this branch's tip: 2,599 passed, which is the base's 2,433 plus the 166 new tests.
  • make typecheck-v2 is clean. mypy on the changed application files outside that gate shows no new errors.
  • ruff check and ruff format --check (0.15.12) are clean.
  • The runtime cache's Redis integration tests pass against a real redis-server.
  • Every Pydantic model schema, the FastAPI OpenAPI with every native router enabled, the /v2/openapi.json preview and the legacy spec are byte-identical to the base. The only additions are adapter entries for the new internal dataclasses.
  • Not fixed here; they're existing test-isolation problems: test_v1_route_contracts.py leaves a fake-backed economy_routes module bound for later tests, and test_alembic_baseline.py's logging setup disables existing loggers for the rest of the run. The new snapshot test records logs from the module logger for that reason.
  • CI: this PR is above the bottom of its stack, so it runs only CI complete, which fails and says why. Its full run starts once the PRs below it merge.

🤖 Generated with Claude Code

@anth-volk
anth-volk added this pull request to stack #3917 October 14, 2026 18:47
@anth-volk anth-volk changed the title ruff complexity economy Refactor the economy and calculation services within ruff's complexity limits Oct 14, 2026
@anth-volk
anth-volk force-pushed the ruff-complexity-economy branch 3 times, most recently from 59307e2 to b87554c Compare October 14, 2026 21:13
@anth-volk
anth-volk force-pushed the ruff-complexity-economy branch 4 times, most recently from 51e6891 to 827680a Compare October 14, 2026 22:56
anth-volk and others added 10 commits October 15, 2026 04:05
Snapshot the complete economy comparison output (UK and US, every
breakdown region and income-change bucket, cliffs) with key order, value
types and float reprs, and add focused characterization tests for the
economy service's request plumbing, the shared-cache keys and records,
the household calculation options and cache reads, SPM selection and
worker-result checks, and the release command.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Split the constituency and local-authority breakdowns into a region
filter, a dataset load, an income-change bucket lookup and one shared
per-area loop, and name the poverty age bands and income-change
thresholds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve an SPM selection in parse, bundle-configuration and
certified-merge steps, check budget-window and annual worker results in
separate helpers, and name the calendar-year length and the release
command's argument count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Also name the in-memory fake's compare-and-delete argument count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Group the policy pair, region, dataset and period into ReformImpactScope
and add ReformImpactIdentity (scope, cache version and options hash), so
the reform-impact cache, its service and the budget-window cache key take
one object instead of eight or more fields. Stored impacts take a
ReformImpactExecution (execution ID, start time, observability ID), and
the error and completion updates drop the scope arguments they ignored.
Cache keys and stored records are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pass an annual or budget-window request as an EconomicImpactRequest,
which lives in its own module so routes need not load the service to
build one. Split the budget-window request into stored-state replay,
batch polling and claimed submission steps, and impact creation into
reuse, stored-impact and claim-and-submit steps. Group the options-hash
bundle fields into RuntimeBundleIdentity and the budget-window year lists
into BudgetWindowYearProgress, build simulation options from the setup
options, and name the number of computing years a progress message lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prepare_household_calculation and calculate_household take a
HouseholdCalculationOptions (add_missing, spm, spm_requested) instead of
three keyword arguments; the calculate route builds it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The per-area rows are plain dicts that the breakdown models validate, so
type them as dict[str, Any] rather than dict, which mypy reads as a
mismatch with the models' field types.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Alembic tests load a logging configuration that disables existing
loggers, so in a full run the comparison module's logger no longer
reached caplog. Attach a recording handler to that logger, enabled for
the test's duration, so the snapshot sees what the module logs in any
test order.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk force-pushed the ruff-complexity-economy branch from 827680a to e26d4a5 Compare October 15, 2026 00:06

This branch was successfully deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Refactor the economy and calculation services within ruff's complexity limits

1 participant