Skip to content

Plan: start the projections from Microcosm's US population (docs; draft #42 registration, not posted) - #517

Merged
MaxGhenis merged 11 commits into
masterfrom
docs/microcosm-start-plan-20261004
Oct 5, 2026
Merged

MaxGhenis merged 11 commits into
masterfrom
docs/microcosm-start-plan-20261004

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

What this adds

Docs only. Nothing here runs.

  • docs/plans/microcosm-start.md is a plan for starting the Social Security projections from Microcosm's US cross-section, the population PolicyEngine-US runs on, instead of the 2011 PSID cohort.
  • docs/plans/microcosm-start-registration-draft.md is a draft issue Candidate 2 design: latent-permanent conditioned chained QRF (research memo) #42 registration of the tests that would score that start: two gates and two report-only checks. It is not posted. Posting waits for Max's go (d947).
  • docs/plans/microcosm-start-evidence/ holds a structure-only script, and its output, for three cached Microcosm releases. It records column lists, source years, support channels and effective sample sizes from household weights. No outcome variable was summarized. The output reproduces gate w1's recorded effective size of 14,327.8 exactly.

The plan in brief

  • Earnings histories. Each frame person gets a history built backward from their start-year earnings rank, with PSID donors (rank-kNN, gate 1's passing method).
    • QRF chains failed gate 1 in both directions: the two-predictor chain forgets (ten-year autocorrelation 0.31–0.37 against 0.54), and versions given a fixed person effect overshoot (0.62–0.67).
    • Three candidates are frozen in the registration, run once together, and adopted by a registered rule: rank-kNN, then rank-kNN with marital strata, then a QRF with the predictors H1 can align across the PSID and the frame.
    • Years the PSID lacks use gate_epuf_fill's certified fills, so H1's run waits for that gate's verdict (d927).
  • Starting stock. People already receiving benefits keep the benefit the frame reports, as the PSID projection does. PIAs are inferred for retired and disabled workers, with named limits.
  • Income and wealth. Taken from Microcosm and mapped to the stage that produces each item: the CPS, the 2015 PUF, the 2022 SCF and the 2023 SIPP. They age by growth plus rank within cell. A1 reports how well that works against PSID panels.
  • Gates.
    • H1 scores the deployed history procedure. One arm uses held-out EPUF careers on a new salted split; the other uses held-out PSID families.
    • Gate 3 gets its first cells: one-year growth against SSA, locked separately once the engine exists.
    • W2 (report-only) checks the starting stock against SSA's Supplement tables.
  • Order and effort. 70–109 agent-days and 22–29.5 review days, both assumptions. The history work can start before funding.
  • Schedule. The plan is reconciled with the NASI 24-month PlanGraph draft, whose months 5–9 are built on the PSID projections and which leaves the Microcosm port out of the 24 months.
  • Risks.
    • Weight concentration: effective size about 14,300 (W1 frame), about 27,000 (Registration 19's) and about 180,000 (a dense build that lacks the needed columns).
    • No education column.
    • 85.6 percent of older beneficiaries are married.
    • A 2024 frame cannot run the 2010 DYNASIM3 tests.
    • PolicyEngine-US takes Social Security benefits as inputs.

Independent review (Opus 5.5 lanes on Subfleet)

Round Head Verdict Report Response
1 24427335 CHANGES REQUIRED (3 blocking, 16 major, 19 minor) review-r1.md response-r1.md
2 26d42793 CHANGES REQUIRED (6 major, 14 minor) review-r2.md response-r2.md
3 b81ff1cd APPROVE AFTER LISTED FIXES review-r3.md response-r3.md
Delta 1 fd4a8ec4 APPROVE review-r3-delta.md three optional follow-ups applied
Delta 2 bb8bacf1 APPROVE review-r4-delta.md —

All reports are in ~/reviews/microcosm-start-plan-20261004/review/. Main changes the reviews forced:

  • the opening-stock statement corrected;
  • candidates frozen together with an adoption rule;
  • a fresh EPUF split, with H1 gating the deployed procedure;
  • gate 3's growth statistic and a runnable bite check;
  • the frame's source_year read as the income year;
  • the dense-build column gap;
  • the forward-law hand-off read as u_w, not the two-year lag.

Decisions

The go to post the registration is queued as d947. The plan's "Decisions needed" also lists:

  • which frame for dynamics;
  • the tolerance multipliers at lock;
  • the 2010 question.

🤖 Generated with Claude Code

Adds a design for starting Microcosm Dynamics' Social Security
projections from Microcosm's US cross-section instead of the PSID, and
a draft issue #42 registration for the tests that would score it. The
registration is not posted; posting waits for Max's go.

The plan covers how each frame person gets an earnings history (rank-kNN
donors, the gate-1 method, against a QRF with every observed predictor),
which Microcosm variables replace the PSID's non-wage income and wealth
and how they age, which gates cover each part and the four new tests
(H1, W2, gate 3's cells, A1), dependencies and tasks, and risks. Docs
only; nothing runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
social-security-model Ready Ready Preview Oct 5, 2026 3:01am UTC

Request Review

MaxGhenis and others added 2 commits October 4, 2026 15:13
The script reads column lists, source years, support channels, age codes
and household-weight effective sample sizes from three cached Microcosm
releases; it summarizes no outcome variable. Its output reproduces gate
w1's recorded effective size (14,327.8) exactly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 3 commits October 4, 2026 15:15
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each becomes a property-based test on invented data: history bounds and
determinism, rank round trips within the clamps, the PIA inference round
trip, weight conservation without entrants, the aging rule's rank
preservation, and the oracle-vs-PolicyEngine-US AIME differential.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lint job failed on the script's formatting. The rewrite passes black
-l 79 and ruff, and no longer lists the table's index as a person column;
every other line of its output is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The independent review (Opus 5.5 on Subfleet) returned CHANGES REQUIRED
with 3 blocking, 16 major and 19 minor findings. Each was checked against
the code and fixed:

- H1's three candidates are frozen in the registration, run once
  together, and adopted by a registered rule (H-A, else H-B, else H-C).
- H1's SSA arm gets its own salted EPUF split instead of reusing
  gate_epuf_fill's TEST, and may learn only from H1-TRAIN.
- Gate 3 scores the signed growth gap against SSA, with its partition
  and constants fixed from the start-year frame before any run, and a
  bite check.
- The opening-stock statement is corrected: the PSID projection already
  carries recorded amounts forward (cola_track_a/benefits.py:747-790).
- The frame's source_year is the income year (2022-2024).
- The dense build lacks the columns the plan needs; decision 2 is
  reworked and the evidence script now lists household columns.
- Properties, forward-law hand-off, PIA-inference limits, disclosures,
  platform and guards are restated or added.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Review round 1: CHANGES REQUIRED, answered.

The independent review is an Opus 5.5 Subfleet lane, at head 24427335. It found 3 blocking, 16 major and 19 minor findings, and confirmed most of the numbers.

The main fixes, at head 26d42793:

  • H1's candidates are frozen in the registration, run once together, and adopted by a registered rule (H-A, else H-B, else H-C).
  • H1's SSA arm has its own salted EPUF split instead of reusing gate_epuf_fill's TEST.
  • Gate 3 scores the signed growth gap against SSA. Its partition and constants are fixed from the start-year frame before any run, and it has a bite check.
  • Opening stock. The statement was backwards and is corrected: the PSID projection already carries recorded amounts forward (cola_track_a/benefits.py:747-790).
  • The frame's source_year is the income year (2022–2024, from the 2023–2025 supplements).
  • The dense Microcosm build lacks the columns the plan needs. Decision 2 is reworked, and the evidence script now lists household columns.

The point-by-point response is in the review record (~/reviews/microcosm-start-plan-20261004/review/response-r1.md). Round 2, a check of each fix, is running. The PR stays a draft until it agrees.

Round 2 (Opus 5.5 on Subfleet) confirmed most round-1 fixes and found
six major items, each checked against the code and fixed:

- H1's SSA arm now gates the deployed procedure (PSID donors, certified
  fills refit on H1-TRAIN, conversion to capped wage-base shares); EPUF
  donors become a report-only configuration.
- Gate 3's bite runs on an invented frame (wrong engine against faithful
  engine), and gate 3 locks separately after the engine exists.
- The forward law reads start-year earnings and u_w, not the two-year
  lag (engine/forward_earnings.py:1444-1453, 697-739); a report-only u_w
  agreement block replaces the false claim.
- P1 is defined on the frame's concept; the PSID-arm bite uses a reserved
  bite part; arm E's partition has a 5 percent false-fail budget.

Minor items (seeds, splits, definitions, disclosure, conventions,
wording, dates) are fixed as listed in the response.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Review round 2: CHANGES REQUIRED, answered at b81ff1cd. Round 2 confirmed most of the round-1 fixes. It raised six major points, all now fixed. N1: H1's SSA arm now gates the deployed procedure (PSID donors, certified fills refit on H1's training part, conversion to capped wage-base shares), with EPUF donors as a report-only configuration. N2: gate 3's bite check runs on an invented frame, and gate 3 locks separately once its engine exists. N3: the forward-law claim is corrected; the law reads start-year earnings and u_w. N4: P1 is defined on the frame's own concept. N5: the PSID arm's bite check uses a reserved bite part. N6: arm E's partition has a 5 percent false-fail budget. The point-by-point response is in ~/reviews/microcosm-start-plan-20261004/review/response-r2.md. A short round-3 confirmation is running.

Round 3 (Opus 5.5 on Subfleet) approved after listed fixes:

- H1's registered run waits for gate_epuf_fill's verdict, which picks
  the fill procedure the deployed configuration refits; the overlap of
  that procedure's selection with H1-TEST is disclosed.
- The anchor ranks within source year and channel.
- The 2012-lag wording matches the code, and the u_w block fixes its
  estimator.
- Arm P's bite margin is priced at the bite sample's size.
- The dense build's missing derived variables are worded precisely.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mention the wait for gate_epuf_fill's verdict where the H1 run is
summarized, list forecasts among the second comment's items, disclose
the EPUF overlap in the plan's H1 summary, and record that PR #512
(selectable baselines) merged on 2026-10-04 as ff3fe30.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis marked this pull request as ready for review October 5, 2026 03:02
@MaxGhenis
MaxGhenis merged commit 03a7d23 into master Oct 5, 2026
12 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merge record. Merged at head bb8bacf1 as 03a7d231 (merge commit, pinned with --match-head-commit). Checks at that head: gh pr checks exited 0 (lint, build, pytest (3.14) and every pytest shard passed). Mergeable: CLEAN. No changes-requested review. Independent review: Opus 5.5 lanes on Subfleet, three rounds plus two delta checks, the last of which approved this head; reports are in ~/reviews/microcosm-start-plan-20261004/review/. This merges a plan and a draft registration only. Posting the registration on issue #42 waits for Max's go (d947).

@MaxGhenis
MaxGhenis deleted the docs/microcosm-start-plan-20261004 branch October 5, 2026 04:14

This branch was successfully deployed

1 active deployment
Preview — bb8bacf1 Deployed Oct 5, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant