Skip to content

Python: match the MATLAB MSSIM formula in Measure_Quality - #783

Merged
AnderBiguri merged 1 commit into
CERN:masterfrom
Dev-next-gen:fix/mssim-matlab-parity
Sep 29, 2026
Merged

AnderBiguri merged 1 commit into
CERN:masterfrom
Dev-next-gen:fix/mssim-matlab-parity

Conversation

@Dev-next-gen

@Dev-next-gen Dev-next-gen commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #782

I was comparing the Python and MATLAB quality measures and noticed that MSSIM does not give the same number on the two sides. Looking at the MSSIM branch of Python/tigre/utilities/Measure_Quality.py against MATLAB/Utilities/Quality_measures/MSSIM.m, four things differ.

The denominators of the luminance, contrast and structure terms use K1 * d ** 2, K2 * d ** 2 and (K2 * d ** 2) / 2, while MSSIM.m squares the whole product in all three: (K1*d)^2, (K2*d)^2 and ((K2*d)^2)/2. The numerators of those same three expressions already use (K1 * d) ** 2 and (K2 * d) ** 2, so the file was inconsistent with itself. On top of that K2 was 0.02 against 0.03 in MSSIM.m (0.03 is also the value used in the original SSIM paper, Wang et al. 2004), and the standard deviations came from np.std(), which normalises by N, while delta, computed a few lines below, and MATLAB's std both normalise by N - 1.

Squaring only K*d leaves the stabilisation term 1/K1 = 100 times too large in l; in c and s, with the K2 difference on top, it is 0.02 * d ** 2 where MSSIM.m has (0.03 * d) ** 2, about 22 times too large. That is enough to keep l, c and s away from 1 even when the two images are identical, and the size of the gap depends on the image, so the numbers are not comparable from one dataset to the next. On master, Measure_Quality(x, x, ["MSSIM"]) * x.size — which should be 1 — gives 0.785 for uniform noise in [0, 1], 0.788 for the same noise scaled to [0, 255], 0.574 for a sparse phantom and 0.799 for a low contrast volume.

The change is confined to the MSSIM branch of that one function; RMSE, nRMSE, CC, UQI and SSD are untouched, and no signature changes. It does move the MSSIM numbers that users get, but in the direction of the MATLAB reference this file is a port of: for the same two inputs, the Python function now returns what MSSIM.m returns. That is parity at the level of the function only; the iterative algorithms do not all pass res_prev and res in the same order on both sides, and d is taken from the first argument.

I checked the change with three tests: the self-similarity identity above, which needs no reference implementation to be convincing, and two comparisons against a literal transcription of MSSIM.m, one with a wide dynamic range and one with d = 1. They were in Python/tests/test_measure_quality_mssim.py in the first version of this PR and were removed on request, so the PR now only touches Measure_Quality.py. All three fail on master and pass with this change. The other Python tests behave the same with and without the change; the only other failure I see in my environment is test_gpuids_default.py::test_awmin_tv_accepts_none, which failed in 1 of 4 full-suite runs with this change and in 1 of 4 on master, and passes on its own. Three files do not collect there either, for reasons unrelated to this change: algorithm_test.py and test_config_gen.py because tigre.demos is not in the installed wheel, and stdout_test.py because it imports the Python 2 StringIO module.

I have not touched UQI, but it has the same np.var() against MATLAB var() normalisation difference, so it is probably worth a separate look.

AI Usage

Found by a defect-hunting pipeline I build and run (Dev-next-gen), using Claude Code with Anthropic's Claude Opus 5.

  • Type of assistance: agentic code analysis and drafting, run by me.
  • Scope: located the divergence between the Python port and MSSIM.m, wrote the patch to Python/tigre/utilities/Measure_Quality.py and the test used to check it (not part of the PR), and drafted this description.
  • How it was found: a systematic comparison of the Python utilities against their MATLAB originals, with every candidate kept only when a test failed before the change and passed after it.
  • Level of modification: the patch and the test were written by the agent; the before/after proof and the Python test suite were run by the pipeline, and the results above are reported as run.

Nothing this pipeline produces is submitted without human approval. The entire pull request — the code change, the tests and this description — was reviewed by a human before the pipeline was allowed to submit it.

(@Dev-next-gen)

@AnderBiguri

Copy link
Copy Markdown
Member

Same as before: remove test, we can merge

The MSSIM branch used K1*d**2 and K2*d**2 in the denominators of the
luminance and contrast terms, and (K2*d**2)/2 in the structure term,
while the numerators of the same expressions already use (K1*d)**2 and
(K2*d)**2. MSSIM.m squares the whole product in every one of those six
places. K2 was also 0.02 here against 0.03 in MSSIM.m, and the standard
deviations came from np.std, which normalises by N, while delta on the
next line and MATLAB's std both normalise by N-1.

Squaring only K*d leaves the stabilisation term 1/K1 = 100 times too
large in l; in c and s, with the K2 difference on top, 0.02*d**2 is
about 22 times the (0.03*d)**2 of MSSIM.m. So l, c and s never reach 1:
on master, MSSIM(x, x) * N is 0.79 for uniform noise and 0.57 for a
sparse phantom instead of 1, and the ceiling depends on the image, so
values are not comparable across datasets.

Fixes CERN#782
@Dev-next-gen
Dev-next-gen force-pushed the fix/mssim-matlab-parity branch from 02e7b7f to 918122b Compare September 29, 2026 11:55
@Dev-next-gen

Copy link
Copy Markdown
Contributor Author

Thanks for merging #781. Same here: the test file is out and the PR now only touches Python/tigre/utilities/Measure_Quality.py (918122b). I also dropped the paragraph about the test from the commit message.

(@Dev-next-gen)

@AnderBiguri
AnderBiguri merged commit 6b0951a into CERN:master Sep 29, 2026
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python MSSIM in Measure_Quality does not match MATLAB's MSSIM.m

2 participants