Skip to content

FIX: Preserve fractional RMSE in polynomial extended-range optimization - #97

Merged
derb12 merged 2 commits into
derb12:developmentfrom
antonsoo:fix/polynomial-optimization-rmse
Oct 11, 2026
Merged

derb12 merged 2 commits into
derb12:developmentfrom
antonsoo:fix/polynomial-optimization-rmse

Conversation

@antonsoo

Copy link
Copy Markdown

Description

optimize_extended_range allocates its squared-error array with the dtype of the search grid. Polynomial degrees are integers, so fractional squared errors are truncated, often producing an all-zero rmse curve and min_rmse for normalized signals. The internal comparison still uses floating-point residuals, so the selected degree can disagree with the minimum of the reported curve.

Allocate the error array as floating point and add signal-scaling regressions for modpoly and imodpoly, with left, right and both-side extensions.

Validation:

  • All six new cases fail before and pass afterward. The full suite passes with 10,018 passed and 113 skipped. Ruff and the full documentation build, including all 23 gallery examples, pass.
  • For 432 fits of measured Raman spectra, independently computed residuals match every corrected error curve and minimum. Selecting a degree from the reported curve disagrees with the residual oracle in 223 cases before and none afterward. All fitted baselines and internally selected degrees remain byte-identical, as do all 216 non-polynomial controls.

The change corrects reported errors; it does not change the optimizer's fitted result.

Type of Pull Request

  • Bug Fix
  • New Feature
  • Miscellaneous Changes (refactor, code improvements, etc.)
  • Documentation or Example Programs

Pull Request Checklist

  • New code and/or documentation is valid for use with the BSD 3-Clause license
  • New code follows the style and docstring guidelines
  • Added/updated tests and ensured they pass locally
  • Verified that documentation builds locally

@derb12

derb12 commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Thanks for finding and fixing this bug! I just have two minor requested changes, otherwise this is good to go.

Comment thread tests/test_optimizers.py Outdated
Comment thread tests/test_optimizers.py
@derb12 derb12 changed the title Preserve fractional RMSE in polynomial extended-range optimization FIX: Preserve fractional RMSE in polynomial extended-range optimization Oct 11, 2026
@derb12
derb12 merged commit f11525d into derb12:development Oct 11, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants