Call the October upgrade's fixed outputs fixed, and build the table's split from a helper

[MaxGhenis]

Follow-up to release dashboard-data-20261010 (#208, merged as 5a8164a). This fixes the two non-blocking nits from its round-2 final review (~/reviews/policybench-haiku-5-5/final-review/round2/review.md).

Changes

1. Upgrades table split comes from a helper. The US references on policyengine-us … (October 2026 upgrade) cell in paper/index.qmd built its parenthetical inline from r.engine_upgrade_restored_count - r.engine_upgrade_restored_ruled_count and r.engine_upgrade_restored_ruled_count. With zero ruled outputs it would print "the 0 ruled on … stay scored", and with one, "the 1 … stay".
   • The new restored_split_clause(restored, ruled, ruling_date) in policybench/paper_results.py, beside restored_split_sentence, builds the whole verb phrase. It handles no fixed outputs, all returned, all ruled, a mix, and singular and plural ("one", "returns", "stays", "output", "defect").
   • The cell calls it through r.engine_upgrade_restored_split_clause.
   • For this release's (18, 4) the output is byte-identical to what the cell printed before.
2. "Fixed", not "restored", where the sentence covers all 18. engine_upgrade_restored_sentence now reads "The fixed outputs take the upstream fixes for …". The four outputs ruled on 2026-10-06 while still scored never left scoring, so "restored" didn't fit them.
   • The docstrings that cover all of them now say "fixed": EngineUpgrade, partition_engine_upgrade_changes, engine_upgrade_restored_count, _fixes, _fix_phrase, _targets and _target_sentence.
   • So do the per-entry refusal messages ("is fixed off its target", "is fixed on a tolerance of"), and the test docstrings.
   • The accessor names stay as they are, so the paper, the card test and the notes test need no rename churn.
   • The note's restored/newlyScored/restoredRuled facts and its "{newlyScored} restored outputs" (the 14 that did return) are already accurate and stay unchanged.

Rendered paper: two words, no numbers

I re-rendered with .venv/bin/python paper/render_paper.py and re-pinned with .venv/bin/python scripts/freeze_snapshot.py --rendered-only. Word-level diffs of the old and new renders:

• HTML: "restored" becomes "fixed" twice (the October paragraph and the table cell). The only other change is the Quarto cell id of the edited table cell.
• PDF (pdftotext): "restored" becomes "fixed" twice. Nothing else changes.
• The manifest and app/src/paperSnapshot.json change only in the pdf and index.html hashes.

Invariants (load-bearing prose helper)

For every 0 ≤ ruled ≤ restored, restored_split_clause:

• names the total in numerals with an agreeing noun and "defect"/"defects";
• says "return" iff restored − ruled > 0, and "stay" and the ruling date iff ruled > 0;
• agrees with restored_split_sentence on both;
• agrees each verb with its count;
• never prints "the 0", "the 1", "0 that", "1 that" or a lone "1" inside the split;
• has exactly one balanced parenthetical;
• refuses ruled > restored and ruled < 0.

Tests

• test_the_split_clause_counts_both_kinds: a Hypothesis property test over (restored ∈ 0..40, ruled ∈ 0..restored) of the invariants above. I also checked the same properties exhaustively over 0..200 by script.
• test_the_fixed_outputs_split_into_returned_and_still_scored pins this release's clause. It also requires the cell to call engine_upgrade_restored_split_clause and refuses the old inline arithmetic.
• test_the_upgrade_sentences_name_what_the_sidecar_records pins "The fixed outputs take the upstream fixes for …" and refuses "restored outputs take" in the source.
• test_rendered_html_calls_the_upgrade_fixes_fixed_not_restored requires the clause and the fixed sentence (twice) in the rendered HTML. test_rendered_pdf_calls_the_upgrade_fixes_fixed_not_restored checks the PDF.
• Mutation check: each of these mutants fails at least one of the new tests: the sentence back to "restored", "the 1 … stay", "1 that … return", and "1 outputs".
• Locally: tests/test_paper_results.py, tests/test_disclosures.py and tests/test_snapshot_artifacts.py give 202 passed and 1 skipped (pre-existing); the full suite is in the notes below. uv run ruff format . and ruff check . are clean.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policybench-site	Ready	Preview	Oct 10, 2026 9:53am UTC