Conversation
The scale fit computed its normalisation constant 1/sum(F_obs^2) via .double(), which raises on MPS: the backend has no float64, and PYTORCH_ENABLE_MPS_FALLBACK covers unimplemented ops, not unsupported dtypes. This broke every test reaching get_scales()/refine_scaler() on the Accelerator CI job, and equally breaks a real TORCHREF_DEVICE=mps refinement. The float64 was unnecessary rather than misplaced. The constant rescales every term of the objective identically, so its own precision cancels out of the minimiser, the gradient direction and the likelihood:penalty ratio; it only has to be finite, positive and O(1)-producing. float32 gives it to ~5e-8 relative on real work sets (torch.sum reduces pairwise, so error grows like log(N)*eps, not N*eps). Also stop the batched direct-summation paths allocating their accumulator as complex128 regardless of configuration, which raises on MPS once max_memory_gb splits the reflection loop and made the batched output dtype disagree with the unbatched one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Ldd5qF6gWrhgHMiv7FcfW1
CIFReader stored each loop under its CIF category, so with parse_all_blocks every data_comp_* block's chem_comp_bond loop overwrote the previous one and only the last block survived. A dictionary defining several compounds restrained one of them and left the rest silently unrestrained. Loops of the same category are now accumulated. Rows carry the block they were read from, because dictionaries that omit comp_id have nothing else to separate their compounds by, and concatenating those without provenance would replace a silent drop with silent cross-contamination. Also fixed, all reachable through the same path: - _filter_by_comp resets the index. _extract_col keeps the source index for a column it finds but returns a fresh RangeIndex for one it does not, so a compound at non-zero index lost the columns that were present. Only reachable once blocks are combined. - builders_fast matched only "positive"/"negative"; CCP4 also writes "positiv"/"negativ", which became NaN and were then dropped outright in builders_numba. - RestraintCIFReader takes verbose, so the read_cif dispatch in readers.py stops raising TypeError on every restraint dictionary. - validate_restraint_data names compounds that come back with no bonds instead of passing over them; an empty table still carries the columns it checked for, which is why this class of failure was silent. Verified against the monomer library rather than transcribed constants: each compound read from a combined file must equal reading it from its own file. 921 compounds over 310 synthesised multi-compound files (bundled + CCP4-9) show no difference. Reverting the fix fails 13 of the 26 new tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018QVRRkhcy5omKhb5a8TM8H
The four shared utilities at paper/ were untracked, two of them named in paper/README.md as steps the reader should run, so the figures could not be reproduced from a clean clone. Same for submit_fig2_arms.py, the canonical Figure-2 submitter, and exF1's grid comparison scripts. Also carries the environment moves the harness already depends on: PHENIX 1.21.1-5286 and node-local CCP4 8.0 (the AFS 7.1 tree is gone), and the torchref scoring array's day/2h limits, which at hour/20min silently timed out the largest models and dropped them from ExtFig 3 without an error. exF4's single-core measurement is updated to the 723-code conserved set. Drops the fcalc results_20260805_124020 tree, superseded on disk by results_20260811_165844 and now covered by .gitignore. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018QVRRkhcy5omKhb5a8TM8H
ScalerBase.forward recomputes _f_sol_raw only when it is None, so rebuilding the
mask without clearing it left F_calc on the old mask. LBFGSRefinement.refine
rebuilt the mask every macrocycle and never cleared it, so with --mode separate
the bulk solvent stayed at the starting coordinates for the whole run: on 1DAW
the F_sol reaching F_calc was 11.5% off in relative L2 norm by cycle 4, while its
total magnitude drifted only 0.07% -- the mask changes shape, not size, which is
why nothing downstream noticed.
The rigid-body and collection paths already paired the two steps correctly. They
are now one operation, ScalerBase.update_solvent, so a caller cannot do half of
it; the collection override is renamed to match. Two kinetic call sites guarded
on hasattr("update_all_solvent") and would have gone quietly dead on the rename.
SolventModel.update_solvent now also drops its own hkl-keyed cache, which holds
the FFT of the mask.
Scoring is unaffected: get_scales() cold-starts through initialize(), so 0-cycle
scores and the final reported R-factor were always built from the current mask.
What was wrong is the coordinate gradient inside the macrocycle loop.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018QVRRkhcy5omKhb5a8TM8H
Merge main changes into dev
Figure 2, exF1-exF5, source data and medians re-run on 0.6.4 (scaler rework plus the F_sol staleness fix). Conserved set n=725, was 723. Paired per-structure against the 0.6.3 archive, PHENIX-scored so the comparison is scorer-independent: TorchRef R-free -0.0018 [-0.0022,-0.0014] 486/252 p=2e-25, R-work -0.0045 [-0.0048,-0.0041] 688/60 p=4e-104, n=748. The scoring effect is separate and ~6x larger. On the UNCHANGED refmac models the TorchRef scorer moves R-free -0.0105 [-0.0112,-0.0100] on 711/767. Null control: the PHENIX scorer on those same models is unchanged on 760/760, so the pipeline moved only what the code changed. exF3's three scorers now agree closely (refmac models: 0.3133 by torchref vs 0.3130 by phenix, was 0.3244); the TorchRef column previously sat ~0.011 high. Figure 2C and exF4 keep the 0.6.3 wall-clock. A results sweep at full parallelism inflates engines unequally -- measured here 5.5x TorchRef, 1.6x PHENIX, 1.0x REFMAC -- which destroys the ratios those panels report. Provenance and the discarded contended numbers are recorded in runs/metrics/RUNTIME_PROVENANCE.md. The F_sol fix itself costs no runtime (matched cells: 294s vs 280s median). exF1 confirms the locked default weights: 3rd of 100 cells by median R-free, 0.0003 off the best, and R-free is flat along the adp axis near the optimum. Unchanged. torchref_score.py called reflection_data.hkl_for_sf(), which 56df759 made private. Every task of the scoring sweep crashed, and torchref_score_array.sh reported rc=0 for all 5308 because `rc=$?` was expanded after a $(date) subshell had reset it, so SLURM logged COMPLETED throughout and stale 0.6.3 scorer JSONs survived --force. Both fixed; the scorer now goes through structure_factors(), which also restores the Friedel conjugation that passing the signed indices skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018QVRRkhcy5omKhb5a8TM8H
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Version 0.6.4
nllto unit-weightlslog_scalewith a Chebyshev polynomialc_isoin sin(theta)/lambdak_sol exp(-ln2 (ss/ss_half)^n), merged sigmoid exponential form