Skip to content

Avoid overflow and underflow of y^2 in the derivatives of x / y - #867

Open
devmotion wants to merge 1 commit into
masterfrom
dw/div-partials
Open

devmotion wants to merge 1 commit into
masterfrom
dw/div-partials

Conversation

@devmotion

Copy link
Copy Markdown
Member

Fixes #866.

For dual numbers x and y, the partials of x / y are computed as (a - (aval / bval) * b) / bval instead of inv(bval) * a - (aval / (bval * bval)) * b. With nested duals, bval is a dual number, so dividing by bval * bval divides by a dual number again, and the n-th derivative divides by y^(2^n). This overflows or underflows long before the derivatives do. For example, the third derivative of x / (x^2 + x^3) is -Inf at x = 1e-30 instead of about -6e120, and the gradient of a Laplace approximation in #866 is -Inf instead of about -386.29. Dividing by bval twice avoids the square. The same issue affects first-order derivatives when y^2 over- or underflows: the derivative of 2x / x is -Inf at x = 1e-200 and 2e-200 at x = 1e200 (instead of 0).

The change is extracted from #858, where it is unrelated to the NaN-safe mode.

Tests:

I bumped the version to 1.4.7.

This pull request was written with the assistance of generative AI (Claude Code).

🤖 Generated with Claude Code

With nested duals, dividing by `bval * bval` divides by a dual number again, so the n-th derivative divides by `y^(2^n)`, which overflows or underflows long before the derivatives do (#866).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@codecov

codecov Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.17%. Comparing base (37a61e1) to head (62a8985).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #867   +/-   ##
=======================================
  Coverage   91.17%   91.17%           
=======================================
  Files          11       11           
  Lines        1077     1077           
=======================================
  Hits          982      982           
  Misses         95       95           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Third-order derivatives of x / y are Inf or NaN for small y, e.g. in gradients of Laplace approximations

1 participant