Skip to content

EPUF-learned career fills: candidates, PSID-2010 application, registered before TEST - #516

Open
MaxGhenis wants to merge 14 commits into
epuf-career-fill-20261003from
epuf-career-fill-candidates-20261004
Open

MaxGhenis wants to merge 14 commits into
epuf-career-fill-20261003from
epuf-career-fill-candidates-20261004

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

What this does

This PR adds the learned replacements for the career assembler's two fill rules, and registers them as gate_epuf_fill's candidates before any TEST read. It is stacked on #515 (the gate's registration), which is stacked on #509. Merge those first.

A gate here is a pass-or-fail test whose rules and thresholds are fixed and published before anything is scored against it.

Family Primary Alternative
Odd years 1997-2011 (now: neighbour mean) odd_forest: two-part random-forest draw per sex. A probability forest for a zero year, a quantile regression forest for the positive share, conditioning on the recorded years around t, age and year. A person-level copula, calibrated on held-out TRAIN persons, carries multi-year persistence odd_knn: kNN triples on t-1, t+1
Years before max(1968, birth_year + 22) (now: zero) pre_donor: rank-kNN donor careers, copying whole blocks from the 3 nearest TRAIN donors of the same sex and birth year pre_chain: chained one-sided draw

Files

  • src/populace_dynamics/estimates/epuf_fill.py: the four fills.
    • Counter-based seeded draws: a person's draw never depends on who else is filled.
    • Byte-reproducible artifacts.
    • load_fill refuses any bytes other than the registered SHA-256.
  • src/populace_dynamics/cohorts/psid2010_epuf_fill.py: fill_careers applies the fills to a built PSID-2010 cohort, opt-in.
    • New provenance values: gap_epuf_drawn and pre_career_epuf_donor.
    • The fills see only recorded career years.
    • career.py, cohorts/psid2010.py and every registered run are untouched.
    • Both new modules are excluded from the birth-evidence reducer's seal, and a test proves the reducer cannot reach them.
  • scripts/fit_epuf_fills.py: fits the four on EPUF TRAIN with the registered parameters.
    • The artifacts are staged outside git, as EPUF is (~/PolicyEngine/epuf-data/fills).
    • runs/epuf_fill_candidates_v1.json pins their SHA-256.
    • A second fit reproduced all four byte for byte.
  • scripts/score_epuf_fill_test.py: the one TEST scoring, through epuf_fill_scoring.score_registered. It refuses until gates.yaml locks the gate.
  • docs/amendments/gate_epuf_fill_candidates_registration.md: the registration, with the DEV prediction stated before TEST.

DEV results (development part; TEST unread)

This is the registered procedure, dry-run on DEV with 20 seeds and the artifacts loaded by hash.

Family Current rule failing Primary Alternative Adopted
odd 100 of 183 improves, 7 failing (mostly at ages 22-29, where the year before the career is hidden from every fill) 46 failing primary, uncertified
pre 131 of 136 certified, 0 failing 50 failing primary

Every DEV score is disclosed in docs/amendments/gate_epuf_fill_dev_scores_*.json*, and the candidate code behind each later score is kept in docs/amendments/gate_epuf_fill_candidate_code/.

Downstream

The DYNASIM projection tests (exercises 1, 3 and 4) build on the PSID-2010 cohort and draw no earnings after 2010, so their AIMEs inherit the career fill. Applying the registered fills to that cohort, as a description and not a registered result, changes the median top-35 indexed earnings through 2010:

Born Pre-career fill
1930-45 +21% (men), +27% (women)
1946-55 +6-7%
1956-65 +10-12%
1966-80 +15-16% (ages 15-21 now count)

The odd-year fill moves medians by between -2% and +4%.

These are registered one-shots, so nothing is rerun here. New runs are queued for Max as decision d929.

Invariants, and where they are tested

  • Fills obey the scoring contract. On their own cells they return finite shares in [0, 1]; every other cell is unchanged; draws are deterministic per seed, differ across seeds, and do not depend on row order. tests/estimates/test_epuf_fill.py, for all four.

  • Artifacts round-trip. Bytes are reproducible, and load_fill refuses other bytes.

  • Donor blocks are real donors' blocks from the same sex and birth year. Years before the first earning age are zero.

  • fill_careers.

    • With the assembler's own rules as fills, it reproduces the assembler's careers.
    • Fills see no pre-career or gap year.
    • Observed rows are untouched, and invalid shares are refused.

    tests/cohorts/test_psid2010_epuf_fill.py.

  • Manifest.

    • Its parameters equal the fit script's.
    • The fill code is unchanged since the fit.
    • Staged files hash to it (skipped where not staged).

    tests/test_epuf_fill_candidates_manifest.py.

Remaining, after Max's ruling on d927

  1. Merge Register gate_epuf against SSA's Earnings Public-Use File (unlocked, report-only) #509, then Register gate_epuf_fill: learned career fills scored on held-out EPUF (awaiting ratification) #515, then this PR.
  2. Flip gates.yaml to lock the gate.
  3. Run scripts/score_epuf_fill_test.py once.
  4. Publish the result and record the adoption in the registration document.

axiom: n/a: microsim-only career imputation; no policy rule is encoded or changed.

Merge method: merge commit only, never squash or rebase. The binding tests diff b722382 (the manifest's code_commit) and cb76ad1 against HEAD. If that commit drops out of master's history, they skip silently instead of failing.

🤖 Generated with Claude Code

@vercel

vercel Bot commented Oct 4, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
social-security-model Ready Ready Preview Oct 4, 2026 7:16pm UTC

Request Review

MaxGhenis added a commit that referenced this pull request Oct 4, 2026
score_registered refuses a candidate spec without a 64-hex SHA-256 and
marks any run with an injected matrix or injected fills as not the
registered TEST scoring. Tests now reach the hash mismatch, check the
oracle's permutation within every stratum, check that the combined
current reading takes the smaller gap on real readings, and show the
neutral rule can only tighten the improves tier. The proposal notes
the two nits in bound files left as they are, the rounding of three
tolerance bounds, and that #516 merges before TEST.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis added a commit that referenced this pull request Oct 4, 2026
fill_careers fills only the gap years the gate scored (1997-2005) by
default and leaves a gap year with no visible neighbour at the
assembler's value; the donor cache is keyed by the bank's content;
persons of uncoded sex take the routed part's copula; a forest leaf
can never be empty; the fit script refuses an existing manifest before
fitting and never replaces a staged file with other bytes, and binds
the gate files it depends on; the score script pins the registered
manifest, records whether its code was clean, leaves a started marker
and publishes no local paths. The manifest is refitted next at this
commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis force-pushed the epuf-career-fill-candidates-20261004 branch from 940ad6f to 2cebe01 Compare October 4, 2026 09:12
MaxGhenis and others added 8 commits October 4, 2026 05:20
estimates/epuf_fill.py holds gate_epuf_fill's candidate fills, fitted on
EPUF TRAIN only: odd years by a two-part random-forest draw (a
probability forest for a zero year, a quantile regression forest for
the positive share) per sex with a person-level copula calibrated on
held-out TRAIN persons, against kNN triples; pre-career years by
rank-kNN donor careers, against a chained one-sided draw. Artifacts are
byte-reproducible npz files staged outside git and loaded by SHA-256.
cohorts/psid2010_epuf_fill.py applies them to a built PSID-2010
cohort's careers behind new provenance values (gap_epuf_drawn,
pre_career_epuf_donor); career.py and the registered runs are untouched.
Both modules are excluded from the birth-evidence reducer's seal.
The DEV log and kept code bytes are updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… by SHA-256)

runs/epuf_fill_candidates_v1.json names the registered fills, fitted at
9f06947 by scripts/fit_epuf_fills.py; a second fit reproduced all four
byte for byte. scripts/score_epuf_fill_test.py is the one TEST scoring,
refused until gates.yaml locks the gate. A DEV dry run of the registered
procedure (20 seeds) predicts: odd primary adopted as an uncertified
improvement (7 of 183 cells failing, at ages 22-29 mostly), pre primary
certified (0 of 136). The registration document records it before TEST.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fill_careers fills only the gap years the gate scored (1997-2005) by
default and leaves a gap year with no visible neighbour at the
assembler's value; the donor cache is keyed by the bank's content;
persons of uncoded sex take the routed part's copula; a forest leaf
can never be empty; the fit script refuses an existing manifest before
fitting and never replaces a staged file with other bytes, and binds
the gate files it depends on; the score script pins the registered
manifest, records whether its code was clean, leaves a started marker
and publishes no local paths. The manifest is refitted next at this
commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e fixed code

The first manifest (SHA-256 8d42153...) recorded the pre-review code
commit. It was never used to read TEST. It is refitted at the reviewed
code next; the artifacts' bytes are expected to be identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nned

runs/epuf_fill_candidates_v1.json is refitted at 598e443 with the code
files clean; the four artifacts' SHA-256 values are unchanged from the
withdrawn manifest. The score script pins this manifest and checks the
lock before it leaves any marker. The registration document records the
code review, the refit and why the DEV dry run still stands.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The score script checks the staged candidates against the manifest
before leaving any marker and binds the cell and wage-base helpers; tests
tie its pin to the manifest and check that it refuses another manifest,
an unlocked gate and an existing result. fill_careers' docstring says
which pre-career years it adds. The two nits left in the pinned fill
module are recorded.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…reachable commit

Rebasing onto #515's fixes left the manifest's code commit unreachable
from the branch. It is refitted at this commit; the artifacts' bytes are
unchanged. The branch syncs with #515 by merge from here on, so the
manifest's commit stays reachable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The four artifacts' SHA-256 values are unchanged. The score script pins
the new manifest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Delta review of #516 (D1): once gates.yaml locks the gate, the refusal
test would have passed the lock check and read TEST. It now patches the
lock status, points --fills-dir at a test folder, and replaces
test_part with a function that fails the test if reached. It also
covers the missing case: gate locked, a staged file with other bytes,
refused before any .started.json marker.

Test-only; no pinned file changes and no refit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Confirmation review of head 7de0644d (independent Opus 5.5 lane, subfleet job 20261004-053206-epuf-fill-review-d1; read-only, no shell). Posted verbatim below the status.

Status

  • 7de0644d fixes D1 from the delta review of bddde85e (job 20261004-052447-epuf-fill-review-delta). It touches only tests/test_epuf_fill_candidates_manifest.py, which no pin covers, so there is no refit.

  • The refusal test now patches the lock status, points --fills-dir at a tmp folder, replaces test_part with a function that fails the test if called, and adds the locked-but-wrong-bytes case (refused, no .started.json).

  • The commands the reviewer could not run, run on the host at 7de0644d: b722382e is an ancestor; the fit code is unchanged since b722382e and the bound files since cb76ad15 (empty diffs); the artifact hashes equal 61349f0b's; the manifest hashes to 83d17a14…6e2c; the targeted tests pass (127), and the tier-count test passes against the full collection (11,504, unchanged).

  • Waits on: Register gate_epuf_fill: learned career fills scored on held-out EPUF (awaiting ratification) #515 (and Register gate_epuf against SSA's Earnings Public-Use File (unlocked, report-only) #509 under it), and d927 (Max ratifies K=1 so the gate can lock). TEST stays unread until the lock.


APPROVE. D1 is fixed. No path through the test can reach test_part or epuf_matrix(TEST), and no other test can read TEST once the gate is locked.

I only had read and search tools, with no shell. So I ran no git commands, hash computations or pytest, and every result the author reports still needs re-running (listed at the end). Everything below is from reading the code.

Findings

No blocking findings. One note, not a defect:

  • N-a: before the second call, the test writes wrong bytes into all four staged files (tests/test_epuf_fill_candidates_manifest.py:166-168). That is correct: whichever file main() checks first fails the hash check, and a missing file can't raise FileNotFoundError instead. It also means the test doesn't separately check that a missing file is refused before the marker. That was never part of the D1 fix.

1. Can any path in the test reach TEST?

The test calls script.main four times, each time with --fills-dir <tmp>/fills (:139). The order of checks in main() is at scripts/score_epuf_fill_test.py:74-121.

Call Lock status Refused at Raises
:141, another manifest ({}) not patched yet manifest SHA check, :78-82, before the lock check ValueError "not the registered 83d1…"
:153, real manifest patched to unlocked (:147-151) lock check, :94-98 TestPartLocked
:170, real manifest, all four staged files wrong patched to locked with REGISTRATION_ID (:158-165) staged-hash check, :102-107, before the marker at :110-111 ValueError "… is not the registered bytes"
:174, output now exists patched to locked output check, :74-75 FileExistsError
  • Every call stops before score_registered. The real gates.yaml is never consulted after the first call. The first call needs no patch, because the manifest refusal comes before the lock check.
  • The real fills can never be found. The fills folder is always the tmp one, so neither ~/PolicyEngine/epuf-data/fills nor POPULACE_DYNAMICS_EPUF_FILLS_DIR is used. That is true whether the real fills are staged or not.
  • The patches have the right signatures. Both patched lock functions are lambda _: …, which matches the positional call at :94.
  • There is a backstop. test_part is replaced with reached_test(**_) (:143-146). score_registered calls g.test_part(gates_path=..., data_dir=...) with keywords only (src/populace_dynamics/harness/epuf_fill_scoring.py:226). That is the only place it reads the matrix (:225-226). So even an unexpected path would fail with AssertionError instead of reading.
  • script.scoring.g is the module score_registered uses.
    • The script imports populace_dynamics.harness.epuf_fill_gate as g and epuf_fill_scoring as scoring (scripts/score_epuf_fill_test.py:29-30).
    • epuf_fill_scoring imports the same epuf_fill_gate as g (epuf_fill_scoring.py:45).
    • The script itself is loaded with spec_from_file_location, but those package imports still go through sys.modules. So script.g, script.scoring.g and the module score_registered reads are one object.
    • score_registered looks up g.test_part when it runs, so the patch takes effect.
    • Patching script.g._gate_lock_status also patches the global that the real test_part checks (epuf_fill_gate.py:1450). That doesn't matter, because test_part itself is replaced.
  • Even the real test_part couldn't read TEST from this test. It would need _TEST_TOKEN to pass epuf_matrix (epuf_fill_gate.py:1373-1377), and nothing here supplies it.

2. Does the new locked-but-wrong-bytes case test the right refusal?

Yes.

  • It gets past every earlier refusal.
    • The output doesn't exist; output.exists() was asserted false at :154.
    • The manifest is the real one, so its SHA matches. That's assuming the pin holds, which test_the_score_script_pins_this_manifest checks.
    • The lock is patched to locked with the correct id.
  • It hits the staged-hash refusal specifically. The match="not the registered bytes" matches only the staged-hash message (:106). The manifest-SHA message ends "not the registered ", so it can't match. If the earlier manifest refusal fired instead, the test would fail rather than pass.
  • It checks no marker is left: :171 asserts there is no result.json.started.json.

3. Can any other test read TEST once the gate is locked?

No.

  • tests/harness/test_epuf_fill_gate.py:
    • :599-614: epuf_matrix(TEST) and epuf_matrix(None) are refused because no token is passed. test_part is called only with tmp gates.yaml files: one unlocked, one locked with the wrong id.
    • :636-662 and :790: these read TRAIN or DEV only, through _fake_epuf, which monkeypatches epuf.read_demographic and epuf.read_annual (:632-633).
  • tests/harness/test_epuf_fill_scoring.py:
    • The calls at :93, :154, :169 and :180 all pass matrix= (:97, :158, :173, :184).
    • The one call without a matrix (:308) passes gates_path= a tmp gates.yaml containing gates: {} (:305-312), so the lock check refuses it.
  • No other test touches this code. Only the seven listed test files import epuf_fill_gate, epuf_fill_scoring or the score script. Of those, test_birth_evidence_artifact.py has no test_part, epuf_matrix, score_registered or TEST reference. Searching every *epuf* test file for test_part, epuf_matrix(, score_registered( and _TEST_TOKEN finds only the uses above.

4. Pinned files

tests/test_epuf_fill_candidates_manifest.py is in none of the pinned lists:

  • the fit script's CODE_FILES (scripts/fit_epuf_fills.py:45-50)
  • the score script's CODE_FILES (scripts/score_epuf_fill_test.py:40-47)
  • BOUND_FILES (scripts/build_epuf_fill_gate_floors.py:73-79)
  • runs/epuf_fill_candidates_v1.json

So a commit that touches only this file can't affect a pin.

What I couldn't verify (needs a shell)

  • That 7de0644d touches only the test file: git show --stat 7de0644d.
  • The author's other reported results:
    • git merge-base --is-ancestor b722382e HEAD
    • the empty diffs since b722382e (fit code) and cb76ad15 (bound files)
    • the artifact sha256 lines matching 61349f0b
    • shasum -a 256 runs/epuf_fill_candidates_v1.json = 83d17a14…6e2c
    • the 127 passed in the targeted pytest run, and the tier-count policy test at 11,504 tests

I made no edits, commits or posts, and read no EPUF microdata.

…s surface

CI shard 1 on #516 failed test__estimator_surface__pins_complete_module_tuple:
the test globs estimates/*.py and expects every module that is not a named
exclusion to be on the registered first-estimates surface. The fills module
is opt-in (only the gate's scoring and its fit script load it), so it gets
its own named exclusion, like the COLA tabulation and Track U modules.
coordinator._ESTIMATOR_SURFACE_SOURCES is unchanged.

Test-only; no pinned file changes and no refit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Head 881f0377: one test-only fix, reviewed APPROVE (independent Opus 5.5 lane, job 20261004-060821-epuf-fill-review-surface; read-only, no shell; posted verbatim after the status).

  • CI at 7de0644d found a second shard-1 failure besides the inherited depletion-cut pin: tests/estimates/test_coordinator.py::test__estimator_surface__pins_complete_module_tuple. It globs estimates/*.py, and the new opt-in estimates/epuf_fill.py is not on the registered first-estimates surface. 881f0377 gives it a named exclusion pinned to exactly that file, as the test already does for the COLA tabulation and Track U modules. coordinator._ESTIMATOR_SURFACE_SOURCES is unchanged, the commit touches only that test file, and no pin covers it, so there is no refit.

  • On the host: test_coordinator.py 68 passed, and the other directory-glob tests (test_anchor_provenance_erratum.py, ss/test_statutory_aime.py, track_u2/*, the registry-compatibility tests, track_a_v2/test_protocol.py) pass.

  • CI is rerunning at 881f0377 (this repo's Tests workflow takes 1-2 hours); I cancelled the superseded runs. Shard 1 will still fail on the inherited pin until Register gate_epuf against SSA's Earnings Public-Use File (unlocked, report-only) #509 merges master.


APPROVE

Commit 881f0377 is a correct fix that changes only a test. Excluding estimates/epuf_fill.py from the first-estimates surface matches the import graph. The test still catches any other new module in estimates/. The test file is not in any pinned or hashed set.

Findings

1. The exclusion is correct. Nothing on the first-estimates path imports epuf_fill.py.

  • Searching src/populace_dynamics/estimates/ for epuf finds only epuf_fill.py itself. So __init__.py, coordinator.py, career.py, runner.py, preparation.py, publication.py, ledgers.py, parameters.py and first_report.py never mention it.
  • src/populace_dynamics/cohorts/psid2010.py has no epuf match.
  • The only importers of populace_dynamics.estimates.epuf_fill in the repo are:
    • src/populace_dynamics/harness/epuf_fill_scoring.py:211, a lazy import inside the function that calls load_fill
    • scripts/fit_epuf_fills.py:38
    • tests: tests/estimates/test_epuf_fill.py:10 and tests/harness/test_epuf_fill_scoring.py:163 (via importorskip)
  • src/populace_dynamics/cohorts/psid2010_epuf_fill.py:37-75 does not import it. Its only first-party import is cola_track_a.statutory, so it takes fills as arguments.
  • Outside their own files, nothing in src/ imports harness/epuf_fill_scoring or harness/epuf_fill_gate. That rules out a transitive route onto the first-estimates path.
  • The new exclusion agrees with how the repo already treats this file. The birth-evidence reducer lists estimates/epuf_fill.py as an opt-in module "nothing historical imports" (scripts/first_estimates_birth_evidence.py:358-361). tests/estimates/test_birth_evidence_artifact.py:470-485 asserts that populace_dynamics.estimates.epuf_fill cannot be reached from the reducer.

2. The comment is accurate (tests/estimates/test_coordinator.py:1541-1543). The gate's scoring loads the fills (epuf_fill_scoring.py:211-214, epuf_fill.load_fill). The fit script imports the module (fit_epuf_fills.py:38). Nothing on the first-estimates path imports it (point 1). scripts/score_epuf_fill_test.py:45 lists the file as a path string for its code-file binding; it does not import it, which fits the comment. One small, harmless gap: the comment doesn't mention psid2010_epuf_fill.py, which applies fills but doesn't import the module.

3. Any other new module still fails the test.

  • The exclusion matches exactly one name: path.name == "epuf_fill.py" (:1544-1546).
  • first_estimates_surface == expected (:1557) fails on any unexcluded new file.
  • epuf_fill_surface == (…/epuf_fill.py,) (:1565-1567) pins the exclusion itself.
  • coordinator._ESTIMATOR_SURFACE_SOURCES == expected (:1556) is unchanged. It still matches src/populace_dynamics/estimates/coordinator.py:60-70 exactly.
  • The older anchor_context_ prefix exclusion is also pinned by an exact tuple (:1568-1574), so a new anchor_context_*.py would still fail.

4. The test file is not pinned or hashed anywhere. Outside tests/, the only file that mentions test_coordinator is .test_durations, which holds pytest timing data and is not a seal. That search covered runs/*.json, scripts/first_estimates_birth_evidence.py (sealed sources), the CODE_FILES lists in scripts/fit_epuf_fills.py and scripts/score_epuf_fill_test.py, and the gate's BOUND_FILES.

What I verified and what I could not

  • Verified by reading: the test body at tests/estimates/test_coordinator.py:1504-1574, _ESTIMATOR_SURFACE_SOURCES, the import searches above, the reducer's exclusion list, the birth-evidence reachability test, and the pinned-set search.
  • Not done: I had no shell, so I could not run git show 881f0377 to confirm the commit touches only this one file. I did not run pytest, so the 68-passed result and the other directory-glob tests rest on the author's host run. The import check was a text search (static, plus the one lazy import). I did not trace imports at runtime.

No files were edited and no EPUF microdata was read.

…ates branch

Brings in master through #509 (492d5a6). Merged, not rebased, so the
manifest's code_commit (b722382) stays on this branch's history; the
manifest, the staged fills and every pinned file are unchanged.

Only the tier-count files conflicted; recounted against the full
collection: unit 6,037, artifact 3,398 (total 11,516).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Head ee1c2f78: synced with master after #509 landed, reviewed APPROVE (independent in-session Opus 5.5 reviewer with a shell; the Subfleet lane was queued on capacity, so I cancelled it; posted verbatim after the status).


Verdicts

Findings

  1. Use a merge commit for both PRs. This is a merge-method caveat, not a defect. The repo allows merge, squash and rebase. If either PR is squashed or rebased, cb76ad15 or b722382e drops out of master's history. CI checks out with fetch-depth: 0 (.github/workflows/tests.yml:24), so a fresh clone may not have those commits. The two binding checks then skip instead of failing, and the binding stops being enforced with no red check:

    • _assert_bound at tests/test_epuf_fill_gate_floors.py:259-265 skips with "the build's commit is not in this clone".
    • test_the_fill_code_is_unchanged_since_the_fit at tests/test_epuf_fill_candidates_manifest.py:73-80 skips the same way.

    Register gate_epuf against SSA's Earnings Public-Use File (unlocked, report-only) #509 itself went in as a merge commit (492d5a60).

  2. CI is not finished. gh pr checks shows pytest-shard (3.14, 2/4) and (4/4) pending on both PRs. Both are MERGEABLE and not drafts.

  3. The brief's master delta is understated. Register gate_epuf_fill: learned career fills scored on held-out EPUF (awaiting ratification) #515's real merge base is 5129ac32, not 65daf03b. Since that base, master also changed docs/design/gate_epuf_block_draft.yaml, scripts/render_gate_epuf_block_draft.py, docs/amendments/gate_epuf_registration_proposal.md, reviews/gate_epuf_round4_confirmation_20261002.md and tests/test_gate_epuf_block_draft.py. None of these overlaps a Register gate_epuf_fill: learned career fills scored on held-out EPUF (awaiting ratification) #515/EPUF-learned career fills: candidates, PSID-2010 application, registered before TEST #516 file, and grepping both PRs' files for those names finds nothing. src/ and gates.yaml are unchanged from 5129ac32 to master, so this doesn't matter.

What I verified

  • Heads and parents match the brief. aa2928d9 has parents 9f2dd835 and 492d5a60; ee1c2f78 has parents 881f0377 and aa2928d9. 65daf03b is an ancestor of master. cb76ad15 is an ancestor of both heads; b722382e is an ancestor of HEAD.

  • Only the tier files were resolved by hand. git show --remerge-diff for both merges touches only tests/tier_counts.json and tests/README-tiers.md. The only files both sides touched since 5129ac32 are those two.

  • The merges left the PR content unchanged. Excluding the tier files, diff(5129ac32..9f2dd835) is byte-identical to diff(origin/master..aa2928d9), and diff(9f2dd835..881f0377) is byte-identical to diff(aa2928d9..ee1c2f78). What was reviewed is what is proposed.

  • The tier counts add up (tiers are assigned per module, tests/conftest.py:89-104).

    Revision unit artifact
    Merge base 5129ac32 5,948 3,361
    Master 5,960 3,361
    aa2928d9 (Register gate_epuf_fill: learned career fills scored on held-out EPUF (awaiting ratification) #515) 6,017 3,388
    ee1c2f78 (EPUF-learned career fills: candidates, PSID-2010 application, registered before TEST #516) 6,037 3,398
  • Collection matches at HEAD (unit 6,037, artifact 3,398, total 11,516) and on a git archive export of aa2928d9 (6,017 / 3,388); the tier-count test passes on both.

  • Targeted tests at HEAD: 234 passed. On the aa2928d9 export: 198 passed, 3 skipped, 3 failed, all three because the export is not a git repo (git rev-parse --show-toplevel exit 128); they pass at HEAD.

  • Pinned and bound files are unchanged: cb76ad15..origin/master on epuf_cells.py, epuf_operator.py, data/epuf.py empty (same from 65daf03b); cb76ad15..aa2928d9 and cb76ad15..HEAD on the five bound files empty; b722382e..HEAD on the fit's code files empty; the manifest hashes to 83d17a14f960033c….

  • Merging rather than rebasing matches repo practice: 118 of the last 200 master commits are merges, 42 of them "merge master into the branch". CLAUDE.md says nothing about merge method.

What I could not verify

  • The git-dependent tests at aa2928d9 exactly (code identical to HEAD, where they pass).
  • The final CI result.

…ratory QRF follow-up

The pre alternative is a one-step binned conditional-quantile chain, not a
QRF: each year conditions on one later share (22 bins), sex and age. The
document now says so where it describes the candidate and its DEV result,
and the TEST procedure requires that wording, so its result is not read
as a QRF result. Its five worst DEV cells are youth levels and zero
shares, which is what the text reports.

Adds an exploratory follow-up after TEST: a full-career microcosm-fit QRF
fill reported beside the registered results. It is not a candidate and
cannot change a tier or adoption.

Doc-only; no registered rule, parameter or pinned file changes. Mechanism
claims checked against epuf_fill.py by an independent reviewer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Head 28428900: doc-only change, independent review APPROVE (in-session Opus 5.5 reviewer with a shell; two rounds).

  • The change. The candidates registration doc now describes pre_chain exactly.
    • It is a one-step binned conditional-quantile chain, not a QRF. Each year conditions on one later share, in 22 bins, plus sex and age.
    • The DEV results and TEST step 5 must describe its result that way. The peer "Microcosm dynamics" asked for this guard so that "the chain failed" is not read as "QRF failed".
    • A new section, "Exploratory follow-up (after TEST, not a candidate)", adds a full-career microcosm-fit QRF fill. It would be reported beside the registered results and cannot change a tier or the adoption.
  • Review, round 1: REQUEST CHANGES.
    • It checked each mechanism claim against epuf_fill.py.
    • It rejected my first draft's claim that the chain "loses persistence". The disclosed DEV log shows its five worst cells are youth levels and zero shares at ages 15-21 (ylevel, yzero) and one pre-career level (plevel), and the log records only the five worst cells. The text now says only what the log shows.
  • Review, round 2: one phrase fixed ("y+2 when y+1 is the career start and a masked odd year").
  • Review, final: APPROVE. The commit touches only that doc, and no registered rule, parameter, pinned file or the manifest changed.
  • Status. CI is rerunning. Register gate_epuf_fill: learned career fills scored on held-out EPUF (awaiting ratification) #515 is green and MERGEABLE at aa2928d9. Both PRs are held for Max's d927 and must be merged with a merge commit.

Max is ruling on d927 (ratify K = 1). The brief he sees includes a DEV-only
sweep of K over the registered dry run, so the sweep is disclosed in the
DEV log before the lock closes it:
- gate_epuf_fill_dev_registered_dryrun.json: the registered TEST procedure
  dry-run on DEV, with every cell (previously summarized in the log only);
- gate_epuf_fill_dev_k_sweep.py and .txt: the re-scoring at K from 0.25 to
  4 with the repository's own score/adoption_tier/adopt/combined_current;
  K = 1 reproduces the record exactly;
- a DEV-log line and a paragraph in the candidates registration.

K = 1 was registered at 14045be, before any DEV score; nothing registered
changes. No pinned file, manifest or bound file is touched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of 0634936 (APPROVE, non-blocking): the log line gave grid-rounded
thresholds as if exact. It now carries the exact breakpoints (odd 0.8968
and 2.6903, odd alternative 3.4117; pre 0.2564 and 0.7693), each a cell's
|gap|/sigma or |gap|/(3 sigma), and the paragraph says the sweep used a
0.005 grid with every breakpoint checked exactly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Head 80eacd41: the DEV sweep of K, shown to Max for d927, is now disclosed. Independent review: APPROVE (in-session Opus 5.5 reviewer with a shell).

What changed

  • Max asked for help deciding d927, which ratifies K = 1. The brief he received includes a DEV-only sweep of K over the registered TEST-procedure dry run. The sweep goes in the DEV log before the lock closes that log:
    • docs/amendments/gate_epuf_fill_dev_registered_dryrun.json: the dry run with every cell. Until now only its summary was logged.
    • docs/amendments/gate_epuf_fill_dev_k_sweep.py and .txt: the dry run re-scored with the repository's own score, adoption_tier, adopt and combined_current.
    • DEV-log line 18, and a paragraph in the candidates registration.
  • The second commit, 80eacd41, follows the review's one note: the log now records the exact thresholds rather than grid-rounded ones.

What the reviewer checked

  • The script's rerun output is byte-identical to the committed .txt.
  • K = 1 reproduces every recorded cell, tier and adoption.
  • The exact breakpoints are odd 0.8968 (improves) and 2.6903 (certified); pre 0.2564 (improves) and 0.7693 (certified). There are no other transitions, so the same primaries are adopted for every K from 0.8968 to 4.
  • K_TOLERANCE = 1.0 predates every DEV score. It is in 45874067 (02:41Z) and 14045be4 (05:01Z), and the first DEV score is at 05:25Z.
  • The dry run's candidate SHA-256s and floors SHA-256 match the registered ones.
  • No pinned or bound file changed, and no test reads the DEV log.

Status: waiting on Max's d927. The brief recommends ratifying K = 1 as registered.

This branch was successfully deployed

1 active deployment
Preview — 80eacd41 Deployed Oct 4, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant