Skip to content

Add probability calibration visualization - #678

Open
ChidiebereNjoku wants to merge 1 commit into
interpretml:mainfrom
ChidiebereNjoku:feat/probability-calibration-625
Open

Add probability calibration visualization#678
ChidiebereNjoku wants to merge 1 commit into
interpretml:mainfrom
ChidiebereNjoku:feat/probability-calibration-625

Conversation

@ChidiebereNjoku

Copy link
Copy Markdown

Description

Adds a CalibrationCurve performance explainer for binary classifiers, following the existing ROC and PR conventions.

The new explainer:

  • computes calibration data using scikit-learn’s calibration_curve
  • supports uniform and quantile binning strategies
  • displays a reliability curve against the perfect-calibration reference
  • includes a predicted-probability density plot
  • is exposed through interpret.perf and included in the performance API documentation

Unit tests cover uniform and quantile calculations, visualization output, public configuration, and invalid arguments.

Validation

  • python -m pytest python/interpret-core/tests/perf/test_curve.py python/interpret-core/tests/visual/test_plot.py -q — 10 passed
  • Ruff formatting check passed
  • Ruff import-order check passed
  • Public CalibrationCurve import verified
  • Documentation notebook JSON validated

Closes #625

Signed-off-by: ChidiebereNjoku <144056487+ChidiebereNjoku@users.noreply.github.com>
@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.36585% with 12 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.33%. Comparing base (560f8dd) to head (6ba60ff).

Files with missing lines Patch % Lines
python/interpret-core/interpret/perf/_curve.py 81.81% 12 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #678      +/-   ##
==========================================
+ Coverage   67.21%   67.33%   +0.11%     
==========================================
  Files          77       77              
  Lines       11735    11813      +78     
==========================================
+ Hits         7888     7954      +66     
- Misses       3847     3859      +12     
Flag Coverage Δ
bdist_linux_311_python 67.07% <85.36%> (+0.09%) ⬆️
bdist_linux_312_python 67.07% <85.36%> (+0.09%) ⬆️
bdist_linux_313_python 67.07% <85.36%> (+0.11%) ⬆️
bdist_linux_314_python 66.97% <85.36%> (+0.10%) ⬆️
bdist_linuxarm_311_python 67.07% <85.36%> (+0.08%) ⬆️
bdist_linuxarm_312_python 67.08% <85.36%> (+0.09%) ⬆️
bdist_linuxarm_313_python 67.08% <85.36%> (+0.09%) ⬆️
bdist_linuxarm_314_python 66.99% <85.36%> (+0.10%) ⬆️
bdist_mac_311_python 67.23% <85.36%> (+0.09%) ⬆️
bdist_mac_312_python 67.23% <85.36%> (+0.09%) ⬆️
bdist_mac_313_python 67.21% <85.36%> (+0.08%) ⬆️
bdist_mac_314_python 67.15% <85.36%> (+0.11%) ⬆️
bdist_win_311_python 67.24% <85.36%> (+0.11%) ⬆️
bdist_win_312_python 67.25% <85.36%> (+0.10%) ⬆️
bdist_win_313_python 67.26% <85.36%> (+0.11%) ⬆️
bdist_win_314_python 67.15% <85.36%> (+0.09%) ⬆️
sdist_linux_311_python 67.03% <85.36%> (+0.11%) ⬆️
sdist_linux_312_python 67.01% <85.36%> (+0.09%) ⬆️
sdist_linux_313_python 67.03% <85.36%> (+0.11%) ⬆️
sdist_linux_314_python 66.92% <85.36%> (+0.10%) ⬆️
sdist_linuxarm_311_python 67.02% <85.36%> (+0.11%) ⬆️
sdist_linuxarm_312_python 67.04% <85.36%> (+0.13%) ⬆️
sdist_linuxarm_313_python 67.04% <85.36%> (+0.11%) ⬆️
sdist_linuxarm_314_python 66.91% <85.36%> (+0.10%) ⬆️
sdist_mac_311_python 67.14% <85.36%> (+0.11%) ⬆️
sdist_mac_312_python 67.14% <85.36%> (+0.09%) ⬆️
sdist_mac_313_python 67.16% <85.36%> (+0.11%) ⬆️
sdist_mac_314_python 67.05% <85.36%> (+0.09%) ⬆️
sdist_win_311_python 67.24% <85.36%> (+0.11%) ⬆️
sdist_win_312_python 67.26% <85.36%> (+0.11%) ⬆️
sdist_win_313_python 67.23% <85.36%> (+0.08%) ⬆️
sdist_win_314_python 67.15% <85.36%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Probability calibration

1 participant