Dev

README.md

@@ -30,7 +30,7 @@ TorchRef is a crystallographic refinement package built entirely on PyTorch. Aut
 
 *Refinement of Phaser-placed AlphaFold models against experimental data, on a conserved set of 723 PDB structures (1.40–3.00 Å). All engines start from the same placed models and are scored by one common validator (PHENIX).*
 
-**(A)** R-work and R-free per engine. Median R-free 0.3197 (TorchRef), 0.3165 (PHENIX), 0.3136 (REFMAC5); paired median difference, TorchRef minus reference, +0.0006 against PHENIX and +0.0047 against REFMAC. **(B)** Geometry RMS Z against REFMAC restraints, ideal 1.0: bond 0.58 (REFMAC 0.58, PHENIX 0.83), main-chain B 1.56 (REFMAC 0.90, PHENIX 1.03). **(C)** Wall-clock runtime per structure on 4 CPU cores: median 0.77 min (REFMAC 0.25, PHENIX 3.09). **(D)** Fraction of the total R-free improvement reached per macrocycle.
+**(A)** R-work and R-free per engine. Median R-free 0.3189 (TorchRef), 0.3164 (PHENIX), 0.3136 (REFMAC5); paired median difference, TorchRef minus reference, +0.0008 against PHENIX and +0.0047 against REFMAC. **(B)** Geometry RMS Z against REFMAC restraints, ideal 1.0: bond 0.58 (REFMAC 0.58, PHENIX 0.83), main-chain B 1.55 (REFMAC 0.90, PHENIX 1.03). **(C)** Wall-clock runtime per structure on 4 CPU cores: median 0.77 min (REFMAC 0.25, PHENIX 3.09). **(D)** Fraction of the total R-free improvement reached per macrocycle.
 
 ## Key Features
 

docs/changelog.rst

@@ -2,6 +2,23 @@ Changelog
 =========
 
 
+Version 0.6.4
+----------
+- Fixed the bulk-solvent ``F_sol`` staying at the starting model's mask for every refinement macrocycle
+- Fixed restraint dictionaries defining several compounds yielding restraints for only one of them
+- Fixed chirality restraints being dropped for the ``positiv``/``negativ`` spellings used by the CCP4 library
+- Compounds that come back with no bond restraints are now reported
+- Fixed written phases (``PH-model``, ``PHWT``, ``PHDELWT``) being negated for reflections whose input Miller indices lay outside the CCP4 ASU
+- Reflections remapped to the CCP4 ASU on load are now reported
+- Switched the scaler's default scale-fit objective from ``nll`` to unit-weight ``ls``
+- Replaced the per-bin ``log_scale`` with a Chebyshev polynomial ``c_iso`` in sin(theta)/lambda
+- Replaced the solvent Debye-Waller factor with ``k_sol exp(-ln2 (ss/ss_half)^n)``, merged sigmoid exponential form
+- Fixed the solvent-mask candidate enumeration, which missed voxels near the atom's grid node
+- Removed the solvent-mask Gaussian smoothing
+- Fixed the scale fit's float64 normalisation constant, which broke MPS
+- Batched direct summation now returns ``dtypes.complex`` instead of always ``complex128``
+
+
 Version 0.6.3
 -------------
 - Fixed peptide-linked residues keeping their free-amino-acid restraint angles

paper/FIGURE_MEDIANS.md

@@ -2,45 +2,45 @@
 
 Auto-generated by `analysis/summarize_medians.py` from the metrics CSVs; numbers match the plotted figures. TorchRef = canonical arm `torchref` (xray 1 / geometry 0.2 / adp 0.02).
 
-## Figure 2A — R-factors (PHENIX-validated) (conserved set, n=723)
+## Figure 2A — R-factors (PHENIX-validated) (conserved set, n=725)
 
 | Model | median R-work | median R-free | n |
 |---|---|---|---|
-| AlphaFold (start) | 0.4079 | 0.4084 | 723 |
-| Refmac | 0.2726 | 0.3136 | 723 |
-| PHENIX | 0.2745 | 0.3165 | 723 |
-| TorchRef | 0.2757 | 0.3197 | 723 |
+| AlphaFold (start) | 0.4078 | 0.4083 | 725 |
+| Refmac | 0.2719 | 0.3135 | 725 |
+| PHENIX | 0.2744 | 0.3161 | 725 |
+| TorchRef | 0.2695 | 0.3165 | 725 |
 
-## Figure 2A — paired ΔR-free, TorchRef − reference (conserved set, n=723)
+## Figure 2A — paired ΔR-free, TorchRef − reference (conserved set, n=725)
 
 Median of the per-structure differences (negative = TorchRef better), with a 10 000-sample bootstrap CI on that median and a Wilcoxon signed-rank p. `Δ of medians` is the difference of the two medians in the table above, shown for contrast: it is *not* the paired statistic and can differ in sign.
 
 | Reference | metric | n | median Δ (paired) | 95% CI | Wilcoxon p | TorchRef better / worse | Δ of medians |
 |---|---|---|---|---|---|---|---|
-| Refmac | R-free | 723 | +0.0047 | [+0.0040, +0.0056] | 6.81e-27 | 226 / 497 | +0.0061 |
-| Refmac | R-work | 723 | +0.0026 | [+0.0017, +0.0035] | 1.93e-16 | 288 / 433 | +0.0031 |
-| PHENIX | R-free | 723 | +0.0006 | [-0.0001, +0.0015] | 5.39e-02 | 336 / 382 | +0.0032 |
-| PHENIX | R-work | 723 | +0.0005 | [-0.0008, +0.0020] | 1.54e-05 | 347 / 374 | +0.0012 |
+| Refmac | R-free | 725 | +0.0030 | [+0.0021, +0.0039] | 1.58e-14 | 266 / 452 | +0.0030 |
+| Refmac | R-work | 725 | -0.0017 | [-0.0027, -0.0010] | 2.02e-01 | 411 / 308 | -0.0024 |
+| PHENIX | R-free | 725 | -0.0010 | [-0.0018, -0.0004] | 7.91e-04 | 398 / 320 | +0.0004 |
+| PHENIX | R-work | 725 | -0.0044 | [-0.0054, -0.0033] | 5.14e-07 | 460 / 264 | -0.0049 |
 
-## Figure 2B — geometry RMSZ (median; ideal = 1.0) (conserved set, n=723)
+## Figure 2B — geometry RMSZ (median; ideal = 1.0) (conserved set, n=725)
 
 | Model | Bond RMSZ | Angle RMSZ | Chiral RMSZ | MC B-factor RMSZ | n |
 |---|---|---|---|---|---|
-| Refmac | 0.58 | 0.95 | 0.54 | 0.90 | 723 |
-| PHENIX | 0.83 | 0.92 | 0.35 | 1.03 | 723 |
-| TorchRef | 0.58 | 0.96 | 0.57 | 1.56 | 723 |
+| Refmac | 0.58 | 0.95 | 0.54 | 0.90 | 725 |
+| PHENIX | 0.83 | 0.92 | 0.35 | 1.03 | 725 |
+| TorchRef | 0.58 | 0.99 | 0.60 | 1.55 | 725 |
 
-## Figure 2C — wall-clock runtime (conserved set, n=723)
+## Figure 2C — wall-clock runtime (conserved set, n=725)
 
 | Engine | median runtime (min) | n |
 |---|---|---|
-| Refmac | 0.25 | 723 |
+| Refmac | 0.25 | 725 |
 | TorchRef | 0.77 | 723 |
-| PHENIX | 3.09 | 723 |
+| PHENIX | 3.09 | 725 |
 
 TorchRef vs Refmac: 3.1× slower; TorchRef vs PHENIX: 4.0× faster.
 
-## Figure 2 — run accounting (N=767 candidates; conserved set n=723)
+## Figure 2 — run accounting (N=767 candidates; conserved set n=725)
 
 Each engine refined the same set of Phaser-placed AlphaFold models. *Validated* = produced a PHENIX-scored R-factor; the headline medians use the conserved intersection where all four engines validated.
 
@@ -49,7 +49,7 @@ Each engine refined the same set of Phaser-placed AlphaFold models. *Validated*
 | AlphaFold (start) | 759 | 8 | 767 |
 | Refmac | 760 | 7 | 767 |
 | PHENIX | 734 | 33 | 767 |
-| TorchRef | 748 | 19 | 767 |
+| TorchRef | 750 | 17 | 767 |
 
 ### Failure reasons (grouped)
 
@@ -62,8 +62,8 @@ Each engine refined the same set of Phaser-placed AlphaFold models. *Validated*
   - 5 × refine: phenix.refine crash: special-position pathology
   - 1 × refine: phenix.refine transient error
   - 1 × refine: phenix.refine: incompatible data flags (Friedel mismatch)
-- **TorchRef** — 19 failed
-  - 11 × refine: TorchRef OOM at 8 GB (large structure)
+- **TorchRef** — 17 failed
+  - 9 × refine: TorchRef OOM at 8 GB (large structure)
   - 8 × scoring: PHENIX validator fails (special-position pathology)
 
 Most failures trace to a **special-position pathology** in some placed AF models: it breaks the uniform PHENIX scorer (so the structure is unscoreable for *every* engine) and makes phenix.refine itself abort, while REFMAC and TorchRef refine it without error. TorchRef's only engine-side failures are OOMs at the 8 GB job limit (large structures; would pass at 16 GB); REFMAC had zero refinement failures.
@@ -82,34 +82,34 @@ The reference CPU is stated because it sets this number: `cpu_xeon6230R` (Cascad
 
 ## Extended Figure 2 — R-factor gap (TorchRef − reference, PHENIX-scored)
 
-n = 723 structures, resolution 1.40–3.00 Å.
+n = 725 structures, resolution 1.40–3.00 Å.
 
 | Reference | median ΔR-work (pp) | median ΔR-free (pp) |
 |---|---|---|
-| PHENIX | +0.05 | +0.06 |
-| REFMAC | +0.26 | +0.47 |
+| PHENIX | -0.44 | -0.10 |
+| REFMAC | -0.17 | +0.30 |
 
 ## Extended Figure 3 — scorer-consistency matrix (median R-free)
 
 | Model | by REFMAC | by PHENIX | by TorchRef | n(paired) |
 |---|---|---|---|---|
-| Refmac | 0.3202 | 0.3130 | 0.3243 | 760 |
-| PHENIX | 0.3247 | 0.3161 | 0.3304 | 734 |
-| TorchRef | 0.3262 | 0.3201 | 0.3334 | 748 |
-| AlphaFold (start) | 0.4158 | 0.4081 | 0.4308 | 759 |
+| Refmac | 0.3202 | 0.3130 | 0.3133 | 760 |
+| PHENIX | 0.3247 | 0.3156 | 0.3180 | 734 |
+| TorchRef | 0.3234 | 0.3170 | 0.3175 | 750 |
+| AlphaFold (start) | 0.4159 | 0.4081 | 0.4105 | 757 |
 
 ## Extended Figure 1 — weight landscape, locked-default cell
 
 Cell geometry=0.215, adp=0.0215 (n=47 structures). NB: the weight screen runs on a fixed ~50-structure subset, so these medians are NOT comparable to the full-benchmark numbers above (different, smaller structure set).
 
 | metric | median |
 |---|---|
-| Bond RMSZ | 0.5455 |
-| Angle RMSZ | 0.9186 |
-| MC B-factor RMSZ | 1.6754 |
-| R-work | 0.2622 |
-| R-free | 0.3039 |
-| R-free − R-work gap | 0.0377 |
+| Bond RMSZ | 0.5833 |
+| Angle RMSZ | 0.9572 |
+| MC B-factor RMSZ | 1.6836 |
+| R-work | 0.2513 |
+| R-free | 0.2985 |
+| R-free − R-work gap | 0.0421 |
 
 ## Extended Figure 4 — single-core runtime (Fig 2c at 1 CPU core)
 

paper/README.md

@@ -18,36 +18,38 @@ The `paper/` directory expects two data sources linked as:
 ```
 paper/
 ├── data -> ../scientific_testing/data            # ~1000 benchmark structures
-│   └── {CODE}/{CODE}.mtz, {CODE}_shaken.pdb, ...
+│   └── {CODE}/{CODE}.mtz, {CODE}.pdb, ...        # reflections + deposited model
 ├── phenix_refinements -> .../refinements         # PHENIX comparison results
 │   └── {CODE}/{CODE}_refined_001.pdb, .log, ...
 ```
 
-**For reviewers:** if cloning fresh, create the data directory and populate it:
+Figure 2 refines **Phaser-placed AlphaFold models**, not the deposited or perturbed
+coordinates: its starting models are `figure2_alphafold_start/placed/{CODE}_af.pdb`, and the
+structure set is the MR-solved subset recorded in `figure2_alphafold_start/mr_status.csv`
+(`solved == 1`, n = 767). Those placed models plus `manifest.json` / `mr_status.csv` are
+*inputs* — produced once by the molecular-replacement pipeline in
+`figure2_alphafold_start/` (`fetch_alphafold.py` → `prepare_search_model.py` →
+`run_mr_one.sh` → `collect_mr_status.py`) and reused by every arm thereafter.
 
-```bash
-cd paper/
+## Figures
 
-# Step 1: Download 1000 structures (PDB + structure factors)
-python figure2_validation/scripts/get_pdb_and_data_files.py
-python figure2_validation/scripts/download_pdb_files.py
+Every `plot_*.py` also writes per-panel **source data** to `source_data/` as it renders — one
+CSV holding the values that panel draws, after all filtering and derivation. See
+[`source_data/README.md`](source_data/README.md) for the file list, units and caveats. There is
+no separate export step: re-render a figure and its CSVs update with it.
 
-# Step 2: Prepare input data
-python figure2_validation/scripts/make_standardized_mtzs.py   # CIF → MTZ
-python figure2_validation/scripts/convert_cifx.py              # CIF → PDB
-python figure2_validation/scripts/shake_cifs.py                # Shake coordinates + B-factors
+### Figure 2: AlphaFold-start Benchmark
+**Location:** `figure2_alphafold_start/`
 
-# Step 3: Create the data symlink (or copy)
-ln -s ../scientific_testing/data data
-```
+TorchRef vs REFMAC5 vs phenix.refine, all four models (including the AlphaFold starting
+point) scored by the same independent validator, `phenix.model_vs_data`. Panels: A
+R-work/R-free, B geometry RMSZ, C wall-clock, D convergence.
 
-## Figures
-
-### Figure 2: Validation Benchmark
-**Location:** `figure2_validation/`
-
-Comparison of TorchRef vs PHENIX refinement across ~1000 PDB structures.
-See `figure2_validation/README.md` for the full reproduction pipeline.
+Submit the three arms with `analysis/submit_fig2_arms.py`, which emits each structure's
+arms adjacently on one pinned CPU model — Panel C compares wall-clock across engines, so
+submitting one engine at a time confounds engine identity with cluster-load window. Then
+`run_af_pipeline.py validate` (REFMAC 0-cycle), the two scoring sweeps, and
+`analysis/aggregate_figure_metrics.py`; render with `plot_figure_af.py`.
 
 ### Figure 3: Performance Benchmarks
 **Location:** `figure3_performance/`
@@ -72,6 +74,43 @@ All input data and refinement outputs included in the directory.
 | ExtFig 1 | AF-start loss-weight landscape (10×10 geometry×adp grid) | Plotting only — uses `figure2_alphafold_start/runs/metrics/weight_grid.csv` |
 | ExtFig 2 | R-factor gap vs resolution (TorchRef − PHENIX & − REFMAC, AF-start, PHENIX-scored) | Plotting only — uses `figure2_alphafold_start/runs/metrics/fig_crossscore.csv` |
 | ExtFig 3 | R-factor scorer consistency (REFMAC / PHENIX / TorchRef cross-scoring) | Plotting only — uses `figure2_alphafold_start/runs/metrics/fig_crossscore.csv` |
-| ExtFig 4 | F_calc splatting optimization breakdown (1DAW) | Requires CPU + GPU benchmarking |
+| ExtFig 4 | Single-core wall-clock runtime, 3 engines (reproduces Fig 2c at 1 CPU core, AF-start, n=715) | Requires re-running all three engines — `exF4/submit_singlecore.py` (2145 cells) then `aggregate_singlecore.py` |
+| ExtFig 5 | X-ray target-mode comparison (`ml` / `ml_noalpha` / `ml_full` / `nll_beta`, AF-start, PHENIX-scored) | Plotting only — uses `figure2_alphafold_start/runs/metrics/mode_*.csv` |
 
 See `extended_figures/instructions.md` for the design document.
+
+### ExtFig 5: reproducing the target-mode comparison
+
+Answers the reviewer question "does the choice of maximum-likelihood variant matter?" Reuses
+Figure 2's exact 2×2 layout via `plot_figure_af.py --prefix/--engines`, so the two figures are
+directly comparable — same scorer (`phenix.model_vs_data`), same conserved-set discipline.
+
+The `ml` cell **is** Figure 2's `torchref` arm; only the other three are extra runs.
+
+```bash
+cd paper/figure2_alphafold_start
+for m in ml_noalpha nll_beta; do
+    ../../.dev/bin/python analysis/submit_local_arm.py --arm mode_$m --xray-mode $m
+done
+# ml_full costs ~4x per gradient (32-node quadrature)
+../../.dev/bin/python analysis/submit_local_arm.py --arm mode_ml_full --xray-mode ml_full \
+    --partition day --time 04:00:00 --mem 16G
+
+# PHENIX-score the new arms (resume-safe; only the missing tasks are emitted)
+../../.dev/bin/python analysis/build_crossscore_worklist.py \
+    --arm ml_noalpha=mode_ml_noalpha --arm ml_full=mode_ml_full --arm nll_beta=mode_nll_beta
+sbatch --array=1-N%60 analysis/crossscore_array.sh
+
+../../.dev/bin/python analysis/aggregate_figure_metrics.py --prefix mode \
+    --arm ml=torchref:torchref --arm ml_noalpha=mode_ml_noalpha:torchref \
+    --arm ml_full=mode_ml_full:torchref --arm nll_beta=mode_nll_beta:torchref
+
+../../.dev/bin/python plot_figure_af.py --prefix mode \
+    -o ../extended_figures/exF5/output/extended_figure5.png \
+    --engines ml_noalpha=ml_noalpha:#2166ac ml=ml:#b2182b \
+              ml_full=ml_full:#762a83 nll_beta=nll_beta:#e08214
+```
+
+Before submitting anything, `../../.dev/bin/python check_submitter_flags.py` asserts every
+submitter still emits flags `refine.py` accepts — a removed or renamed flag otherwise kills
+several thousand jobs at argparse, one at a time.